-- Logs begin at Fri 2022-06-03 15:30:30 UTC, end at Fri 2022-06-03 16:40:30 UTC. -- Jun 03 16:01:04.575933 dv-cinder-836996-28-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 03 16:01:04.584413 dv-cinder-836996-28-iscsi devstack@keystone.service[80276]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 03 16:01:04.586965 dv-cinder-836996-28-iscsi devstack@keystone.service[80276]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Jun 3 16:01:04 2022] *** Jun 03 16:01:04.586965 dv-cinder-836996-28-iscsi devstack@keystone.service[80276]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 03 16:01:04.586965 dv-cinder-836996-28-iscsi devstack@keystone.service[80276]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 03 16:01:04.586965 dv-cinder-836996-28-iscsi devstack@keystone.service[80276]: nodename: dv-cinder-836996-28-iscsi Jun 03 16:01:04.586965 dv-cinder-836996-28-iscsi devstack@keystone.service[80276]: machine: x86_64 Jun 03 16:01:04.586965 dv-cinder-836996-28-iscsi devstack@keystone.service[80276]: clock source: unix Jun 03 16:01:04.586965 dv-cinder-836996-28-iscsi devstack@keystone.service[80276]: pcre jit disabled Jun 03 16:01:04.586965 dv-cinder-836996-28-iscsi devstack@keystone.service[80276]: detected number of CPU cores: 4 Jun 03 16:01:04.586965 dv-cinder-836996-28-iscsi devstack@keystone.service[80276]: current working directory: / Jun 03 16:01:04.586965 dv-cinder-836996-28-iscsi devstack@keystone.service[80276]: detected binary path: /usr/bin/uwsgi-core Jun 03 16:01:04.587542 dv-cinder-836996-28-iscsi devstack@keystone.service[80276]: your processes number limit is 47680 Jun 03 16:01:04.587542 dv-cinder-836996-28-iscsi devstack@keystone.service[80276]: your memory page size is 4096 bytes Jun 03 16:01:04.587542 dv-cinder-836996-28-iscsi devstack@keystone.service[80276]: detected max file descriptor number: 2048 Jun 03 16:01:04.587542 dv-cinder-836996-28-iscsi devstack@keystone.service[80276]: lock engine: pthread robust mutexes Jun 03 16:01:04.587542 dv-cinder-836996-28-iscsi devstack@keystone.service[80276]: thunder lock: enabled Jun 03 16:01:04.587542 dv-cinder-836996-28-iscsi devstack@keystone.service[80276]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 03 16:01:04.587542 dv-cinder-836996-28-iscsi devstack@keystone.service[80276]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 03 16:01:04.631092 dv-cinder-836996-28-iscsi devstack@keystone.service[80276]: Python main interpreter initialized at 0x55e39a3079c0 Jun 03 16:01:04.631092 dv-cinder-836996-28-iscsi devstack@keystone.service[80276]: python threads support enabled Jun 03 16:01:04.631092 dv-cinder-836996-28-iscsi devstack@keystone.service[80276]: your server socket listen backlog is limited to 100 connections Jun 03 16:01:04.631092 dv-cinder-836996-28-iscsi devstack@keystone.service[80276]: your mercy for graceful operations on workers is 90 seconds Jun 03 16:01:04.631447 dv-cinder-836996-28-iscsi devstack@keystone.service[80276]: mapped 403077 bytes (393 KB) for 2 cores Jun 03 16:01:04.631530 dv-cinder-836996-28-iscsi devstack@keystone.service[80276]: *** Operational MODE: preforking *** Jun 03 16:01:04.632308 dv-cinder-836996-28-iscsi devstack@keystone.service[80276]: *** uWSGI is running in multiple interpreter mode *** Jun 03 16:01:04.632308 dv-cinder-836996-28-iscsi devstack@keystone.service[80276]: spawned uWSGI master process (pid: 80276) Jun 03 16:01:04.632502 dv-cinder-836996-28-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 03 16:01:04.633345 dv-cinder-836996-28-iscsi devstack@keystone.service[80276]: spawned uWSGI worker 1 (pid: 80286, cores: 1) Jun 03 16:01:04.633450 dv-cinder-836996-28-iscsi devstack@keystone.service[80276]: spawned uWSGI worker 2 (pid: 80287, cores: 1) Jun 03 16:01:04.635383 dv-cinder-836996-28-iscsi devstack@keystone.service[80276]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 03 16:01:10.405355 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 03 16:01:10.405355 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: from cryptography.utils import int_from_bytes Jun 03 16:01:10.406116 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 03 16:01:10.406116 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: from cryptography.utils import int_from_bytes Jun 03 16:01:10.406246 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 03 16:01:10.406246 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: from cryptography.utils import int_from_bytes Jun 03 16:01:10.406246 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 03 16:01:10.406246 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: from cryptography.utils import int_from_bytes Jun 03 16:01:10.455578 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 03 16:01:10.455772 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 03 16:01:10.455953 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 03 16:01:10.456049 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] command line args: [] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 03 16:01:10.456154 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 03 16:01:10.456237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 03 16:01:10.456352 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] command line args: [] {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 03 16:01:10.456434 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ================================================================================ {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 03 16:01:10.456628 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] admin_token = **** {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.456705 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 03 16:01:10.456811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ================================================================================ {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 03 16:01:10.456894 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] config_dir = [] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.457069 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] admin_token = **** {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.457151 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.457250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] config_source = [] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.457330 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] config_dir = [] {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.457434 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] debug = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.457510 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.457635 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.457879 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] config_source = [] {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.457879 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] debug = True {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.458000 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.458058 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.458220 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.458324 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.458417 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.458516 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.458659 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.458771 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.458900 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] list_limit = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.459007 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.459108 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] log_config_append = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.459212 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] list_limit = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.459335 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.459436 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] log_config_append = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.459537 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] log_dir = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.459683 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.460282 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] log_file = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.460394 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] log_dir = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.460488 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.460578 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] log_file = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.460668 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.460783 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.460900 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.461011 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.461125 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.461234 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.461234 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.461456 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.461581 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.461694 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.461804 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.461919 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.462030 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.462139 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.462258 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.462375 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.462485 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.462599 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.462714 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.462832 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.462941 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.463059 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.463169 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.463284 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.463430 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.463542 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.463542 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.463752 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.463869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.463982 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.41/identity {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.464134 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] publish_errors = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.464254 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.41/identity {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.464369 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.464488 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] publish_errors = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.464603 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.464713 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.464821 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.464930 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.465037 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.465157 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.465327 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.465440 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.465559 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] standard_threads = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.465675 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.465793 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.465910 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] standard_threads = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.466020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.466136 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.466253 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.466364 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.466481 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] use_journal = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.466598 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.466708 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] use_json = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.466824 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] use_journal = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.466940 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] use_stderr = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.467050 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] use_json = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.467170 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] use_syslog = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.467283 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] use_stderr = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.467416 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.467536 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] use_syslog = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.467648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.467758 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 16:01:10.467874 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.467991 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.468101 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.468210 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.468325 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.468444 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.468554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.468663 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.468772 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.468881 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.468995 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.469181 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.469293 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.469404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.469512 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.469621 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.469729 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.469849 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.469965 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.470075 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.470180 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.470296 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.470407 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.470532 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.470643 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.470752 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.470869 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.470980 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.471089 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.471207 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.471331 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.471443 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.471554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.471653 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.471744 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.471851 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.471969 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.472080 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.472195 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.472305 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.472420 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.472538 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.472649 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.472793 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.472908 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.473018 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.473142 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.473262 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.473373 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.473481 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.473597 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.473713 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.473823 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.473940 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.474050 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.474159 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.474273 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.474388 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.474503 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.474689 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.474802 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.474910 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.475025 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.475135 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.475250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.475439 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.475552 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.475663 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.475771 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.475880 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.475996 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.476106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.476215 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.476324 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.476432 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.476540 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.476656 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.476768 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.476877 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.476985 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.477094 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.477202 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.477319 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.477435 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.477545 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.477653 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.477768 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.477885 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.477987 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.478122 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.478223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.478315 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.478407 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.478498 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.478600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.478694 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.478785 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.478876 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.478991 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.479091 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.479207 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.479331 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.479471 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.479624 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.479740 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.479858 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.479980 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.480092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.480207 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.480319 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.480428 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.480537 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.480645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.480755 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.480863 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.480974 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.481083 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.481192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.481310 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.481430 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.481542 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.481659 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.481776 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.481897 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.482006 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.482124 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.482236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.482344 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.482456 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.482572 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.482674 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.482783 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.482901 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.483013 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.483123 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.483233 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.483383 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.483497 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.483619 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.483732 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.483842 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.483952 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.484052 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.484167 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.484289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.484406 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.484517 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.484636 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.484754 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.484866 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.484984 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.485097 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.485207 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.485317 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.485427 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.485537 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.485723 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.485723 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.485933 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.486044 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.486155 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.486273 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.486385 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.486494 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.486604 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.486604 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.486815 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.486934 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.487046 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.487046 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.487251 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.487376 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.487487 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.487612 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.487731 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.487842 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.487953 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.488068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.488180 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.488299 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.488423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.488546 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.488658 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.488768 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.488878 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.488988 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.489098 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.489215 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.489329 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.489439 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.489557 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.489669 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.489779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.489900 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.490013 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.490137 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.490335 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.490452 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.490563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.490686 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.490799 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.490909 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.491019 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.491129 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.491239 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.491394 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] auth.external = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.491515 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.491627 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.491737 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.491847 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.491958 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] auth.external = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.492077 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.492216 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.492343 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] auth.password = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.492466 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.492579 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] auth.token = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.492690 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.492807 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.492919 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] auth.password = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.493037 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.493151 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] auth.token = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.493261 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.493373 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.493485 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.493605 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.493717 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.493828 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.493944 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.494057 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.494167 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.494286 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.494409 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] credential.caching = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.494522 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.494633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.494744 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.494855 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.494974 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] credential.caching = True {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.495086 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.495198 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.495322 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.495436 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.495546 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.495670 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.495783 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.495894 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.496004 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.496115 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.496225 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.496345 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.496463 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.496575 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.496687 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.496799 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.496910 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.497037 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.497150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.497261 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.497375 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.497487 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.497605 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.497727 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.497845 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.497947 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.498057 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] federation.caching = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.498168 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.498279 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.498397 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.498510 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.498621 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] federation.caching = True {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.498733 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.498844 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.498954 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.499142 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.499268 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.499430 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.499542 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.499654 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.499777 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.499902 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.500016 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.500127 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.500237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.500348 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.500496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.500626 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.500740 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.500852 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.500962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.caching = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.501073 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.501184 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.501307 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.501421 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.501532 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] identity.caching = True {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.501642 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.501642 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.501852 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.501971 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.502084 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.502195 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.502307 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.502417 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.502527 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.502646 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.502759 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.502870 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.502981 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.503092 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.503211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.503341 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.503465 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.503578 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.503689 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.503800 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.503919 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.504033 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.504145 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.504261 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.504377 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.504489 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.504609 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.504722 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.504840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.504952 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.505064 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.505175 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.505298 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.505411 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.505522 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.505633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.505744 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.505854 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.505974 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.506088 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.506199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.506309 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.506420 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.507088 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.507216 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.507383 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.507498 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.507609 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.507720 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.507832 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.507955 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.508069 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.508181 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.508292 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.508403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.508513 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.508636 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.508749 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.508860 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.508971 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.509082 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.509192 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.509311 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.509424 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.509543 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.509656 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.509767 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.509878 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.510002 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.510115 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.510226 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.510338 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.510448 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.510558 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.510677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.510790 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.510900 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.511011 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.511121 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.511232 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.511370 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.511486 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.511598 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.511715 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.511827 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.511938 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.512058 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.512170 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.512282 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.512394 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.512505 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.512615 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.512735 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.512848 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.512960 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.513071 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.513182 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.513293 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.513414 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.513527 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.513639 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.513750 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.513861 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.user = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.513972 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.514091 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.514204 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.514319 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.514431 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.514544 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.514654 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.514773 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.514886 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.514997 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.515107 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.515219 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.515365 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.515480 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.515592 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.515704 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.515815 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.515926 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.516046 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.516159 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.516270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.516381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.516492 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.516604 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.516722 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.516836 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.516946 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.517058 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.517169 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.517280 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.517401 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.517401 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.517626 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.517738 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.517849 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.517959 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.518078 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.518190 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.518301 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.518412 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.518524 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.518636 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.518755 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.518867 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.518978 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.519088 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.519199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.519324 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.519449 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.519569 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.519682 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.519793 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.519904 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.520015 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.520134 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.520246 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.520358 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.520475 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.520588 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.520699 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.520820 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.520932 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.521050 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.521162 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.521273 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.521384 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] resource.caching = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.521564 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.521680 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.521793 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.521904 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.522015 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.522156 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.522792 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.522908 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.523019 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] resource.caching = True {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.523130 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.523241 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.523400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.523526 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.523639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.523751 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.523862 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.523974 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.524085 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.524204 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.524317 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] role.caching = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.524433 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.524544 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] role.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.524654 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.524765 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.524884 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.524997 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.525109 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.525220 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.525332 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] role.caching = True {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.525443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.525563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] role.driver = sql {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.525676 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.525787 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.525897 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.526010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.526121 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.526241 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.526354 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.526466 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.526576 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.526687 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.526798 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.526918 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.527032 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.527143 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.527254 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.527382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.527506 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.527620 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.527730 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.527892 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.527892 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.528105 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.528223 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.528337 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.528447 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.528558 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.528669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.528780 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.528780 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.529258 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.529368 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.529498 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.529611 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.529722 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.529845 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.529958 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.530068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.530180 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.530290 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.530401 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.530520 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.530633 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.530744 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.530854 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.530965 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.531075 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.531194 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.531326 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.531439 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.531550 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.531660 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.531771 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.531893 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.532006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.532117 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.532228 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.532339 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.532449 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.532568 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] token.caching = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.532681 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.532791 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.532901 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.533011 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.533130 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.533242 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.533354 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.533465 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.533576 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] token.caching = True {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.533686 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.533805 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.533918 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.534034 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.534146 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.534257 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.534372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.534492 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.534605 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.534605 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.534812 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.534922 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.535033 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.535152 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.535264 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.535446 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.535558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.535669 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.535779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.535901 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.536015 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.536125 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.536236 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.536236 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.536442 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.536562 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.536674 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.connection = **** {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.536798 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.536919 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.537033 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.537143 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.537255 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.537365 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.537365 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.537585 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.537696 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.537839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] database.connection = **** {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.537961 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.538082 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.538195 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.538314 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.538427 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.538538 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.538650 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.538761 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.538872 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.538991 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.539104 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.539215 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.539343 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.539457 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.539568 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.539693 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.539807 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.539918 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.540030 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.540141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.540252 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.540436 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.540552 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.540665 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.540777 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.540777 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 03 16:01:10.540986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.540986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.541204 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.541344 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.541590 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.541831 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 16:01:10.542004 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80286) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 03 16:01:10.570045 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.570273 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.570469 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80286) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.570552 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.570705 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80286) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.570935 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80286) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.578991 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.579476 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80286) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.607710 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.607937 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.608035 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80286) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.608246 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80286) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.662280 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.662577 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80286) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.711632 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.711868 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.712043 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80286) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.712281 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80286) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.778565 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80286) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.778691 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.778822 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80286) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.779181 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.779257 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80286) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.779351 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.779462 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80286) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.779543 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.779685 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80286) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.779768 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.779904 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80286) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.779985 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.802779 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.803017 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.803104 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80286) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.803247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.803844 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80286) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.804005 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.804086 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80286) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.804223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.804306 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80286) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.804437 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.804517 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80286) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.804649 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.804739 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80286) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.804891 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80286) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.815958 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.816262 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80286) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.829066 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.829301 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.829525 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.829605 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80286) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.829753 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.829880 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80286) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.829965 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.830056 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80286) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.830166 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.830259 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80286) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.830379 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.830459 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80286) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.830658 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80286) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.830878 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80286) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 16:01:10.847546 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.847758 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 03 16:01:10.847942 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.848096 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.848178 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 03 16:01:10.848298 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 03 16:01:10.848478 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.848558 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.848679 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 03 16:01:10.848807 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.848949 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.849060 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.849208 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.849313 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.849466 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.849723 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.860352 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.860629 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.860777 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.860904 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.861056 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.861188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.861327 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.861614 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.869813 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 16:01:10.870264 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80286) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 16:01:10.873133 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 16:01:10.873411 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.873586 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80286) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 16:01:10.873690 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.873864 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.873971 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.874147 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.874256 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.874431 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.874623 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.874711 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.874902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.875073 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.875194 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.875377 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.875506 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.875666 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.875780 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.875940 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.876047 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.876231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.876512 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.903157 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.904241 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.905081 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 16:01:10.905359 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.905882 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80286) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 16:01:10.906164 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.910683 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 16:01:10.910969 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.911249 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.911533 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80286) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 16:01:10.911822 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.912103 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.918618 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 16:01:10.918898 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.919529 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80286) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 16:01:10.919817 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.923576 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 16:01:10.923862 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.924143 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.924426 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.924518 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80286) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 16:01:10.924700 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.924781 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.924973 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.925053 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.925244 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.925325 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.925511 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.925592 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.925802 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.926072 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.926338 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.945317 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.945592 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.946110 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.946387 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.950107 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 16:01:10.950954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80286) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 16:01:10.953753 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.954049 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.954598 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.954898 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.959017 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 16:01:10.959964 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80286) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 16:01:10.962673 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 16:01:10.963572 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80286) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 16:01:10.967773 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.968073 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.968357 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.968643 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.968723 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.968920 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.969003 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.969201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.969282 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.969513 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.969793 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.970078 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.989437 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80286) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 16:01:10.989722 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.990092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.990372 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.990683 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 16:01:10.991519 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.991890 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.992166 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:10.998816 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.000065 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.000575 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.000851 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.001785 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.002061 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.003932 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80286) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 16:01:11.004215 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.004497 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.004782 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.005082 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.005161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 16:01:11.005405 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.005685 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.005974 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.006269 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.019119 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80286) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 16:01:11.019427 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.019713 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.019995 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.020216 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 16:01:11.020307 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.020501 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.020583 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.020784 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.020864 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.021061 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.021337 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.021607 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.021880 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.041340 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 16:01:11.041444 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80286) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 16:01:11.044591 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 16:01:11.044680 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80286) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 16:01:11.047916 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.048003 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.049528 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.049627 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.051173 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 16:01:11.051255 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80286) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 16:01:11.051474 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.051568 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.051750 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.051831 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.059163 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.059247 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.060824 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 16:01:11.060919 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80286) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 16:01:11.064039 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.064122 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.064327 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.064408 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.064610 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.064703 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.064885 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.064964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.074126 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 16:01:11.074209 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80286) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 16:01:11.074411 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.074493 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.074690 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.074771 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.082601 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 16:01:11.082696 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80286) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 16:01:11.082885 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.082966 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.083165 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.083246 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.083461 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.083555 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.083729 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.083810 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.084010 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.084091 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.084281 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.084363 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.084553 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.085129 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.085330 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.085411 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.085597 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.085675 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.085863 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.085950 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.086129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.086208 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.086401 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.086479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.086667 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80286) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.086753 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 16:01:11.124918 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55e39a3079c0 pid: 80287 (default app) Jun 03 16:01:11.125139 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55e39a3079c0 pid: 80286 (default app) Jun 03 16:01:11.128326 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1226859-3efd-4693-891d-830880964018 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:11.128624 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1226859-3efd-4693-891d-830880964018 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:11.128905 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1226859-3efd-4693-891d-830880964018 None None] PATH_INFO: `/v3` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:11.133608 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1/1] 10.222.0.41 () {50 vars in 791 bytes} [Fri Jun 3 16:01:11 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:19.086066 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d4883-80f8-4d86-8f28-df6fbb23f024 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:19.086628 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d4883-80f8-4d86-8f28-df6fbb23f024 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:19.086628 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d4883-80f8-4d86-8f28-df6fbb23f024 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:19.090887 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1/2] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:19 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:01:19.096058 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e970f-7196-44a8-9707-ecf22d5446a0 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:19.096483 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e970f-7196-44a8-9707-ecf22d5446a0 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:19.096634 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e970f-7196-44a8-9707-ecf22d5446a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:19.166223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: INFO dbcounter [None req-028e970f-7196-44a8-9707-ecf22d5446a0 None None] Registered counter for database keystone Jun 03 16:01:19.187811 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG oslo_db.sqlalchemy.engines [None req-028e970f-7196-44a8-9707-ecf22d5446a0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80287) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 03 16:01:19.194562 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writer thread running {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 03 16:01:19.473786 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG passlib.handlers.bcrypt [None req-028e970f-7196-44a8-9707-ecf22d5446a0 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80287) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 03 16:01:19.474246 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG passlib.handlers.bcrypt [None req-028e970f-7196-44a8-9707-ecf22d5446a0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80287) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 03 16:01:19.529906 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-028e970f-7196-44a8-9707-ecf22d5446a0 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:19.531534 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-028e970f-7196-44a8-9707-ecf22d5446a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:19.653607 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 2/3] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:19 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 559 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 03 16:01:19.820551 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d4883-80f8-4d86-8f28-df6fbb23f024 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:19.888341 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: INFO dbcounter [None req-180d4883-80f8-4d86-8f28-df6fbb23f024 None None] Registered counter for database keystone Jun 03 16:01:19.907591 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG oslo_db.sqlalchemy.engines [None req-180d4883-80f8-4d86-8f28-df6fbb23f024 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80286) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 03 16:01:19.913647 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writer thread running {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 03 16:01:20.134150 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d4883-80f8-4d86-8f28-df6fbb23f024 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:20.138305 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f693e-08c9-4012-a6af-332a767eeff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:20.138955 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f693e-08c9-4012-a6af-332a767eeff6 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:20.139240 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f693e-08c9-4012-a6af-332a767eeff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:20.139546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f693e-08c9-4012-a6af-332a767eeff6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:20.147466 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f693e-08c9-4012-a6af-332a767eeff6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:20.198535 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: WARNING keystone.common.rbac_enforcer.enforcer [None req-096f693e-08c9-4012-a6af-332a767eeff6 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 03 16:01:20.198876 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f693e-08c9-4012-a6af-332a767eeff6 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:20.208070 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 2/4] 10.222.0.41 () {58 vars in 1324 bytes} [Fri Jun 3 16:01:19 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 16:01:20.214030 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e970f-7196-44a8-9707-ecf22d5446a0 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:20.228969 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e970f-7196-44a8-9707-ecf22d5446a0 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:20.233425 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ce0ee-19e4-47e9-819e-525ada45dbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:20.234530 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ce0ee-19e4-47e9-819e-525ada45dbee admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:20.234821 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ce0ee-19e4-47e9-819e-525ada45dbee admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:20.235102 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ce0ee-19e4-47e9-819e-525ada45dbee admin admin] PATH_INFO: `/v3/projects/450b64d0f10f4cb8949a26d9eea4aef4` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:20.236659 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2ce0ee-19e4-47e9-819e-525ada45dbee admin admin] RBAC: Authorizing `identity:get_project(project_id=450b64d0f10f4cb8949a26d9eea4aef4)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:20.288209 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9e2ce0ee-19e4-47e9-819e-525ada45dbee admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 03 16:01:20.288564 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2ce0ee-19e4-47e9-819e-525ada45dbee admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:20.293234 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 3/5] 10.222.0.41 () {56 vars in 1207 bytes} [Fri Jun 3 16:01:20 2022] GET /identity/v3/projects/450b64d0f10f4cb8949a26d9eea4aef4 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:22.519796 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab901a0-dc16-42f8-b348-68c8383aafe3 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:22.520276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab901a0-dc16-42f8-b348-68c8383aafe3 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:22.520276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab901a0-dc16-42f8-b348-68c8383aafe3 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:22.525038 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 3/6] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:01:22.530654 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b08ff0-c78d-4ac6-a54f-a31c4f9bbebe None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:22.531059 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b08ff0-c78d-4ac6-a54f-a31c4f9bbebe None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:22.531215 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b08ff0-c78d-4ac6-a54f-a31c4f9bbebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:22.549506 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-d4b08ff0-c78d-4ac6-a54f-a31c4f9bbebe None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:22.551002 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d4b08ff0-c78d-4ac6-a54f-a31c4f9bbebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:22.558730 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 4/7] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 03 16:01:22.580880 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab901a0-dc16-42f8-b348-68c8383aafe3 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:22.593568 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab901a0-dc16-42f8-b348-68c8383aafe3 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:22.597731 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad154f0-c315-40c5-a91c-464482f8a118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:22.598357 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad154f0-c315-40c5-a91c-464482f8a118 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:22.598649 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad154f0-c315-40c5-a91c-464482f8a118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:22.598929 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad154f0-c315-40c5-a91c-464482f8a118 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:22.606561 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad154f0-c315-40c5-a91c-464482f8a118 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:22.608351 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad154f0-c315-40c5-a91c-464482f8a118 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:22.616938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 4/8] 10.222.0.41 () {58 vars in 1324 bytes} [Fri Jun 3 16:01:22 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 16:01:22.621908 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b08ff0-c78d-4ac6-a54f-a31c4f9bbebe None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:22.630575 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b08ff0-c78d-4ac6-a54f-a31c4f9bbebe None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:22.635430 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbbb886-1d71-402f-ab6f-2ecf9a077e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:22.636272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbb886-1d71-402f-ab6f-2ecf9a077e41 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:22.636552 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbb886-1d71-402f-ab6f-2ecf9a077e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:22.636826 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbb886-1d71-402f-ab6f-2ecf9a077e41 admin admin] PATH_INFO: `/v3/users/5aca81affd5542cf9480c6e76c8c5478` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:22.638189 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbbb886-1d71-402f-ab6f-2ecf9a077e41 admin admin] RBAC: Authorizing `identity:get_user(user_id=5aca81affd5542cf9480c6e76c8c5478)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:22.640032 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbbb886-1d71-402f-ab6f-2ecf9a077e41 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:22.641559 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 5/9] 10.222.0.41 () {56 vars in 1198 bytes} [Fri Jun 3 16:01:22 2022] GET /identity/v3/users/5aca81affd5542cf9480c6e76c8c5478 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:24.937508 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa5f89-95a1-4946-9bf8-76e2a012ccd6 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:24.938024 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa5f89-95a1-4946-9bf8-76e2a012ccd6 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:24.938121 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa5f89-95a1-4946-9bf8-76e2a012ccd6 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:24.939040 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422483d6-1fed-43a9-89d3-7ab5e25e8a71 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:24.939438 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422483d6-1fed-43a9-89d3-7ab5e25e8a71 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:24.939509 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 5/10] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:01:24.939642 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422483d6-1fed-43a9-89d3-7ab5e25e8a71 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:24.940966 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 6/11] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 16:01:24.944395 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f50f03-adfa-4096-a7a4-11aab9efd187 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:24.944685 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f50f03-adfa-4096-a7a4-11aab9efd187 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:24.944972 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f50f03-adfa-4096-a7a4-11aab9efd187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:24.946248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2913b0c-e4ae-412c-86ad-923d673e7651 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:24.946541 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2913b0c-e4ae-412c-86ad-923d673e7651 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:24.946819 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2913b0c-e4ae-412c-86ad-923d673e7651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:24.963323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-e2913b0c-e4ae-412c-86ad-923d673e7651 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:24.964511 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e2913b0c-e4ae-412c-86ad-923d673e7651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:24.985775 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG passlib.handlers.bcrypt [None req-63f50f03-adfa-4096-a7a4-11aab9efd187 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80286) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 03 16:01:24.986192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG passlib.handlers.bcrypt [None req-63f50f03-adfa-4096-a7a4-11aab9efd187 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80286) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 03 16:01:25.032674 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-63f50f03-adfa-4096-a7a4-11aab9efd187 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:25.034227 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-63f50f03-adfa-4096-a7a4-11aab9efd187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:25.041955 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 7/12] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 16:01:25.063778 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2913b0c-e4ae-412c-86ad-923d673e7651 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:25.087149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:25.145331 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 6/13] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:01:25.168613 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2913b0c-e4ae-412c-86ad-923d673e7651 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:25.169889 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f50f03-adfa-4096-a7a4-11aab9efd187 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:25.201849 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7af9d0-2d76-4b23-9d35-395cd3c6899d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:25.202421 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7af9d0-2d76-4b23-9d35-395cd3c6899d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:25.202705 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7af9d0-2d76-4b23-9d35-395cd3c6899d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:25.202982 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7af9d0-2d76-4b23-9d35-395cd3c6899d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:25.269018 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f50f03-adfa-4096-a7a4-11aab9efd187 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:25.269240 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7af9d0-2d76-4b23-9d35-395cd3c6899d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:25.270726 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7af9d0-2d76-4b23-9d35-395cd3c6899d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:25.271311 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:25.294345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:25.306453 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc553cc-bb34-4555-a30c-da34144671f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:25.307517 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc553cc-bb34-4555-a30c-da34144671f3 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:25.307806 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc553cc-bb34-4555-a30c-da34144671f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:25.308084 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc553cc-bb34-4555-a30c-da34144671f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:25.319768 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc553cc-bb34-4555-a30c-da34144671f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:01:25.330808 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-ccc553cc-bb34-4555-a30c-da34144671f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:01:25.330808 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:01:25.330808 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:01:25.330808 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:01:25.330808 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:01:25.330808 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:01:25.330808 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:01:25.330808 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:01:25.330808 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:01:25.330808 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:01:25.330808 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:01:25.330808 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:01:25.330808 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:01:25.330808 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:01:25.330808 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:01:25.330808 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:01:25.331709 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:01:25.331709 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:01:25.331709 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:01:25.331709 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:01:25.331709 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:01:25.331709 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:01:25.331709 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:01:25.331709 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:01:25.331709 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:01:25.331709 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:01:25.331709 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:01:25.331709 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:01:25.331709 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:01:25.331709 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:01:25.331709 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:01:25.331709 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:01:25.332486 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:01:25.332486 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:01:25.332486 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:01:25.332486 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:01:25.332486 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:01:25.332486 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:01:25.332486 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:01:25.332486 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:01:25.332486 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:01:25.332486 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:01:25.332486 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:01:25.332486 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:01:25.332486 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 7/14] 10.222.0.41 () {56 vars in 1096 bytes} [Fri Jun 3 16:01:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:01:25.338365 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc553cc-bb34-4555-a30c-da34144671f3 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:25.368562 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 8/15] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:01:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 03 16:01:25.373947 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7af9d0-2d76-4b23-9d35-395cd3c6899d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:25.436255 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc553cc-bb34-4555-a30c-da34144671f3 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:25.470737 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df3e5b-a5ae-408e-afb4-635507598fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:25.471387 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df3e5b-a5ae-408e-afb4-635507598fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:25.471673 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df3e5b-a5ae-408e-afb4-635507598fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:25.471950 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df3e5b-a5ae-408e-afb4-635507598fd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:25.472562 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7af9d0-2d76-4b23-9d35-395cd3c6899d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:25.472707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30df3e5b-a5ae-408e-afb4-635507598fd4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:25.473016 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30df3e5b-a5ae-408e-afb4-635507598fd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:25.474460 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30df3e5b-a5ae-408e-afb4-635507598fd4 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:25.486134 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 8/16] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:01:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:01:25.491569 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df3e5b-a5ae-408e-afb4-635507598fd4 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:25.496705 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:25.509149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ded109c-8814-46a7-ab1e-054ed9e8bbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:25.509943 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded109c-8814-46a7-ab1e-054ed9e8bbba None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:25.510232 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded109c-8814-46a7-ab1e-054ed9e8bbba None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:25.510514 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded109c-8814-46a7-ab1e-054ed9e8bbba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:25.518762 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:25.520649 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ded109c-8814-46a7-ab1e-054ed9e8bbba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:25.522676 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ded109c-8814-46a7-ab1e-054ed9e8bbba None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:25.543260 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-5ded109c-8814-46a7-ab1e-054ed9e8bbba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:01:25.543260 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:01:25.543260 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:01:25.543260 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:01:25.543260 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:01:25.543260 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:01:25.543260 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:01:25.543260 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:01:25.543260 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:01:25.543260 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:01:25.543260 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:01:25.543260 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:01:25.543260 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:01:25.543260 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:01:25.543260 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:01:25.544254 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:01:25.544254 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:01:25.544254 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:01:25.544254 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:01:25.544254 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:01:25.544254 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:01:25.544254 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:01:25.544254 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:01:25.544254 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:01:25.544254 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:01:25.544254 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:01:25.544254 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:01:25.544254 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:01:25.544254 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:01:25.544254 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:01:25.544254 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:01:25.544254 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:01:25.545066 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:01:25.545066 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:01:25.545066 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:01:25.545066 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:01:25.545066 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 9/17] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:01:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:01:25.550255 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ded109c-8814-46a7-ab1e-054ed9e8bbba None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:25.590982 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df3e5b-a5ae-408e-afb4-635507598fd4 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:25.622864 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd5bab-a7c6-4a9a-be68-3aedc8361ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:25.623761 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd5bab-a7c6-4a9a-be68-3aedc8361ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:25.624045 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd5bab-a7c6-4a9a-be68-3aedc8361ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:25.624319 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd5bab-a7c6-4a9a-be68-3aedc8361ca7 None admin] PATH_INFO: `/v3/users/5aca81affd5542cf9480c6e76c8c5478` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:25.625837 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bd5bab-a7c6-4a9a-be68-3aedc8361ca7 None admin] RBAC: Authorizing `identity:get_user(user_id=5aca81affd5542cf9480c6e76c8c5478)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:25.627278 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bd5bab-a7c6-4a9a-be68-3aedc8361ca7 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:25.628936 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 9/18] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:01:25 2022] GET /identity/v3/users/5aca81affd5542cf9480c6e76c8c5478 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:01:25.634786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd5bab-a7c6-4a9a-be68-3aedc8361ca7 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:25.646817 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ded109c-8814-46a7-ab1e-054ed9e8bbba None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:25.680316 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea8774-79cc-4150-b027-893ff0cda7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:25.680966 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea8774-79cc-4150-b027-893ff0cda7b6 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:25.681249 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea8774-79cc-4150-b027-893ff0cda7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:25.681525 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea8774-79cc-4150-b027-893ff0cda7b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:25.682254 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ea8774-79cc-4150-b027-893ff0cda7b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:25.682561 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ea8774-79cc-4150-b027-893ff0cda7b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:25.683967 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ea8774-79cc-4150-b027-893ff0cda7b6 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:25.701862 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 10/19] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:01:25.728006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:25.735627 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd5bab-a7c6-4a9a-be68-3aedc8361ca7 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:25.772694 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a25cfea-e50d-4110-b8fa-3640d9994df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:25.773512 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a25cfea-e50d-4110-b8fa-3640d9994df7 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:25.773798 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a25cfea-e50d-4110-b8fa-3640d9994df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:25.774077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a25cfea-e50d-4110-b8fa-3640d9994df7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:25.775327 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a25cfea-e50d-4110-b8fa-3640d9994df7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:25.776792 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a25cfea-e50d-4110-b8fa-3640d9994df7 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:25.778598 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 10/20] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:01:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:01:25.784826 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea8774-79cc-4150-b027-893ff0cda7b6 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:25.821476 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:25.889813 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea8774-79cc-4150-b027-893ff0cda7b6 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:25.923769 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2656510f-d941-4e0c-95b9-15d24e1b99de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:25.924664 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2656510f-d941-4e0c-95b9-15d24e1b99de None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:25.924953 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2656510f-d941-4e0c-95b9-15d24e1b99de None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:25.925233 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2656510f-d941-4e0c-95b9-15d24e1b99de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:25.926072 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2656510f-d941-4e0c-95b9-15d24e1b99de None admin] RBAC: Adding query filter params (role.id=75936a608b8149099ff54e317f3a6740, scope.domain.id=default, user.id=5aca81affd5542cf9480c6e76c8c5478) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:25.926389 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2656510f-d941-4e0c-95b9-15d24e1b99de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:25.927867 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2656510f-d941-4e0c-95b9-15d24e1b99de None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:25.935069 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 11/21] 10.222.0.41 () {56 vars in 1322 bytes} [Fri Jun 3 16:01:25 2022] GET /identity/v3/role_assignments?user.id=5aca81affd5542cf9480c6e76c8c5478&scope.domain.id=default&role.id=75936a608b8149099ff54e317f3a6740 => generated 233 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:28.286662 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ee773-ffbc-4a92-8b27-02169d36f10f None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:28.288214 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ee773-ffbc-4a92-8b27-02169d36f10f None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:28.288696 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ee773-ffbc-4a92-8b27-02169d36f10f None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:28.290378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 11/22] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:01:28.296450 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d231a4-7127-435d-81d8-15a1e4e301a1 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:28.297474 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d231a4-7127-435d-81d8-15a1e4e301a1 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:28.298019 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d231a4-7127-435d-81d8-15a1e4e301a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:28.322597 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b5d231a4-7127-435d-81d8-15a1e4e301a1 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:28.324218 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b5d231a4-7127-435d-81d8-15a1e4e301a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:28.410595 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 12/23] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:01:28.447088 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ee773-ffbc-4a92-8b27-02169d36f10f None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:28.596053 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ee773-ffbc-4a92-8b27-02169d36f10f None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:28.637508 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632115ef-2e27-4872-8f3c-9c9325c49403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:28.638658 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632115ef-2e27-4872-8f3c-9c9325c49403 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:28.639175 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632115ef-2e27-4872-8f3c-9c9325c49403 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:28.639682 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632115ef-2e27-4872-8f3c-9c9325c49403 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:28.666178 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632115ef-2e27-4872-8f3c-9c9325c49403 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:28.668057 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632115ef-2e27-4872-8f3c-9c9325c49403 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:28.692917 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f4bcc-04bb-4314-b04c-c8c6aedaa00c None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:28.693512 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-632115ef-2e27-4872-8f3c-9c9325c49403 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:01:28.693512 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:01:28.693512 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:01:28.693512 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:01:28.693512 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:01:28.693512 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:01:28.693512 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:01:28.693512 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:01:28.693512 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:01:28.693512 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:01:28.693512 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:01:28.693512 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:01:28.693512 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:01:28.693512 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:01:28.693512 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:01:28.694398 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:01:28.694398 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:01:28.694398 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:01:28.694398 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:01:28.694398 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:01:28.694398 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:01:28.694398 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:01:28.694398 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:01:28.694398 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:01:28.694398 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:01:28.694398 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:01:28.694398 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:01:28.694398 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:01:28.694398 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:01:28.694398 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:01:28.694398 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:01:28.694398 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:01:28.695185 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:01:28.695185 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:01:28.695185 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:01:28.695185 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:01:28.695185 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 12/24] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:01:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:01:28.695974 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f4bcc-04bb-4314-b04c-c8c6aedaa00c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:28.696465 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f4bcc-04bb-4314-b04c-c8c6aedaa00c None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:28.698044 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 13/25] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:01:28.700774 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632115ef-2e27-4872-8f3c-9c9325c49403 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:28.704783 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e99bd-e783-4b04-bae5-ff6f21c0c56f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:28.705307 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e99bd-e783-4b04-bae5-ff6f21c0c56f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:28.705806 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e99bd-e783-4b04-bae5-ff6f21c0c56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:28.721829 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:28.732830 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-315e99bd-e783-4b04-bae5-ff6f21c0c56f None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:28.734035 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-315e99bd-e783-4b04-bae5-ff6f21c0c56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:28.775797 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:28.864306 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 14/26] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 16:01:28.902536 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632115ef-2e27-4872-8f3c-9c9325c49403 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:28.907959 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e99bd-e783-4b04-bae5-ff6f21c0c56f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:28.966531 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb3fa7-71ae-4683-8100-ac0a96ba11eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:28.967769 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb3fa7-71ae-4683-8100-ac0a96ba11eb None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:28.969008 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb3fa7-71ae-4683-8100-ac0a96ba11eb None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:28.970110 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb3fa7-71ae-4683-8100-ac0a96ba11eb None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:28.971245 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eb3fa7-71ae-4683-8100-ac0a96ba11eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:28.973059 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eb3fa7-71ae-4683-8100-ac0a96ba11eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:28.976673 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eb3fa7-71ae-4683-8100-ac0a96ba11eb None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:29.014477 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 13/27] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:01:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:29.023721 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc00f06-0678-42f3-a1be-7c7d31ec6b78 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:29.024674 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc00f06-0678-42f3-a1be-7c7d31ec6b78 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:29.025167 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc00f06-0678-42f3-a1be-7c7d31ec6b78 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:29.030344 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 14/28] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:01:29.037387 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc00f06-0678-42f3-a1be-7c7d31ec6b78 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:29.128404 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e99bd-e783-4b04-bae5-ff6f21c0c56f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:29.173420 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc6872-37de-4d86-9c6f-0ba7f41c04c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:29.177016 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc6872-37de-4d86-9c6f-0ba7f41c04c5 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:29.177526 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc6872-37de-4d86-9c6f-0ba7f41c04c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:29.178067 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc6872-37de-4d86-9c6f-0ba7f41c04c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:29.197953 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:29.206077 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abc6872-37de-4d86-9c6f-0ba7f41c04c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:01:29.218323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-5abc6872-37de-4d86-9c6f-0ba7f41c04c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:01:29.218323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:01:29.218323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:01:29.218323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:01:29.218323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:01:29.218323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:01:29.218323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:01:29.218323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:01:29.218323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:01:29.218323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:01:29.218323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:01:29.218323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:01:29.218323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:01:29.218323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:01:29.218323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:01:29.218323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:01:29.219272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:01:29.219272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:01:29.219272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:01:29.219272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:01:29.219272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:01:29.219272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:01:29.219272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:01:29.219272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:01:29.219272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:01:29.219272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:01:29.219272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:01:29.219272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:01:29.219272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:01:29.219272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:01:29.219272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:01:29.219272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:01:29.220068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:01:29.220068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:01:29.220068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:01:29.220068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:01:29.220068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:01:29.220068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:01:29.220068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:01:29.220068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:01:29.220068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:01:29.220068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:01:29.220068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:01:29.220068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:01:29.220068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 15/29] 10.222.0.41 () {56 vars in 1096 bytes} [Fri Jun 3 16:01:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:01:29.221419 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38033a72-169b-4d42-8928-8c109f97f932 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:29.221419 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38033a72-169b-4d42-8928-8c109f97f932 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:29.221419 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38033a72-169b-4d42-8928-8c109f97f932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:29.245356 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc00f06-0678-42f3-a1be-7c7d31ec6b78 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:29.264803 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-38033a72-169b-4d42-8928-8c109f97f932 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:29.266024 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-38033a72-169b-4d42-8928-8c109f97f932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:29.288742 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798cec56-2515-4f12-b579-6d2df574c74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:29.289407 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798cec56-2515-4f12-b579-6d2df574c74a None admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:29.289694 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798cec56-2515-4f12-b579-6d2df574c74a None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:29.289968 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798cec56-2515-4f12-b579-6d2df574c74a None admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:29.291482 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798cec56-2515-4f12-b579-6d2df574c74a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:29.292933 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798cec56-2515-4f12-b579-6d2df574c74a None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:29.296863 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:29.376589 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 16/30] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:01:29.377525 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38033a72-169b-4d42-8928-8c109f97f932 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:29.483270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38033a72-169b-4d42-8928-8c109f97f932 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:29.483507 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 15/31] 10.222.0.41 () {60 vars in 1141 bytes} [Fri Jun 3 16:01:29 2022] POST /identity/v3/projects => generated 301 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:01:29.484957 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f58edb-8a5e-4ee5-9903-f92f989de9ea None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:29.485447 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f58edb-8a5e-4ee5-9903-f92f989de9ea None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:29.485874 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f58edb-8a5e-4ee5-9903-f92f989de9ea None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:29.487652 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 16/32] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:01:29.488661 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f58edb-8a5e-4ee5-9903-f92f989de9ea None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:29.529957 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af606ef2-7be7-4c8a-b278-2b161304b9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:29.531051 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af606ef2-7be7-4c8a-b278-2b161304b9e8 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:29.531560 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af606ef2-7be7-4c8a-b278-2b161304b9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:29.532030 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af606ef2-7be7-4c8a-b278-2b161304b9e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:29.533802 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af606ef2-7be7-4c8a-b278-2b161304b9e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:29.534668 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af606ef2-7be7-4c8a-b278-2b161304b9e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:29.536707 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af606ef2-7be7-4c8a-b278-2b161304b9e8 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:29.553890 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 17/33] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:01:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:29.556029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ac67f-bdd9-43ee-920d-2564b7dd6f89 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:29.556579 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ac67f-bdd9-43ee-920d-2564b7dd6f89 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:29.557047 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ac67f-bdd9-43ee-920d-2564b7dd6f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:29.595188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-740ac67f-bdd9-43ee-920d-2564b7dd6f89 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:29.596460 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-740ac67f-bdd9-43ee-920d-2564b7dd6f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:29.610654 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:29.682535 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f58edb-8a5e-4ee5-9903-f92f989de9ea None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:29.711279 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 18/34] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:01:29.712014 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:29.712409 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ac67f-bdd9-43ee-920d-2564b7dd6f89 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:29.721387 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a9ffa-0c2f-4552-9f1f-ee13802240da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:29.722073 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a9ffa-0c2f-4552-9f1f-ee13802240da None admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:29.722366 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a9ffa-0c2f-4552-9f1f-ee13802240da None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:29.722654 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a9ffa-0c2f-4552-9f1f-ee13802240da None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:29.724605 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4a9ffa-0c2f-4552-9f1f-ee13802240da None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:29.726261 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4a9ffa-0c2f-4552-9f1f-ee13802240da None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:29.749474 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 17/35] 10.222.0.41 () {60 vars in 1131 bytes} [Fri Jun 3 16:01:29 2022] POST /identity/v3/roles => generated 219 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:01:29.751107 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d43ae-0f55-4d03-bd57-b78f4fa356b5 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:29.751438 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d43ae-0f55-4d03-bd57-b78f4fa356b5 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:29.751780 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d43ae-0f55-4d03-bd57-b78f4fa356b5 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:29.753194 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 18/36] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:01:29.754135 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d43ae-0f55-4d03-bd57-b78f4fa356b5 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:29.847857 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ac67f-bdd9-43ee-920d-2564b7dd6f89 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:29.881683 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d43ae-0f55-4d03-bd57-b78f4fa356b5 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:29.891516 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8a531-32db-4c3e-b4cb-d5d0e2a2aa4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:29.892994 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8a531-32db-4c3e-b4cb-d5d0e2a2aa4d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:29.893290 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8a531-32db-4c3e-b4cb-d5d0e2a2aa4d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:29.893576 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8a531-32db-4c3e-b4cb-d5d0e2a2aa4d None admin] PATH_INFO: `/v3/users/5aca81affd5542cf9480c6e76c8c5478` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:29.895570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa8a531-32db-4c3e-b4cb-d5d0e2a2aa4d None admin] RBAC: Authorizing `identity:get_user(user_id=5aca81affd5542cf9480c6e76c8c5478)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:29.898136 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa8a531-32db-4c3e-b4cb-d5d0e2a2aa4d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:29.901072 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 19/37] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:01:29 2022] GET /identity/v3/users/5aca81affd5542cf9480c6e76c8c5478 => generated 247 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:29.907377 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72766a09-5958-48c4-bb82-7024a077cfe6 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:29.907777 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72766a09-5958-48c4-bb82-7024a077cfe6 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:29.909080 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72766a09-5958-48c4-bb82-7024a077cfe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:29.932092 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-72766a09-5958-48c4-bb82-7024a077cfe6 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:29.932278 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b253e-d643-4c77-b78e-8233de34b59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:29.932945 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b253e-d643-4c77-b78e-8233de34b59b None admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:29.933295 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b253e-d643-4c77-b78e-8233de34b59b None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:29.933391 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-72766a09-5958-48c4-bb82-7024a077cfe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:29.933576 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b253e-d643-4c77-b78e-8233de34b59b None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:29.934588 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4b253e-d643-4c77-b78e-8233de34b59b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:29.940139 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4b253e-d643-4c77-b78e-8233de34b59b None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:29.942647 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:29.968262 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 19/38] 10.222.0.41 () {60 vars in 1131 bytes} [Fri Jun 3 16:01:29 2022] POST /identity/v3/roles => generated 225 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:01:29.969978 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b253e-d643-4c77-b78e-8233de34b59b None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:30.047563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:30.049104 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 20/39] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:01:30.087424 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72766a09-5958-48c4-bb82-7024a077cfe6 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:30.130193 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b253e-d643-4c77-b78e-8233de34b59b None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:30.180693 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e32e0f-82c1-44ae-84e8-5de23648829e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:30.181573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e32e0f-82c1-44ae-84e8-5de23648829e None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:30.181862 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e32e0f-82c1-44ae-84e8-5de23648829e None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:30.182147 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e32e0f-82c1-44ae-84e8-5de23648829e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:30.183606 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e32e0f-82c1-44ae-84e8-5de23648829e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:30.185087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e32e0f-82c1-44ae-84e8-5de23648829e None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:30.186936 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 20/40] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:01:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:30.192163 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f5d7d-315b-4a6e-bbee-71c628fd87e7 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:30.192163 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f5d7d-315b-4a6e-bbee-71c628fd87e7 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:30.192163 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f5d7d-315b-4a6e-bbee-71c628fd87e7 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:30.192163 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 21/41] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:01:30.196324 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f5d7d-315b-4a6e-bbee-71c628fd87e7 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:30.242486 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72766a09-5958-48c4-bb82-7024a077cfe6 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:30.278660 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a1f56-5c12-4dde-8490-1644a989e085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:30.279740 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a1f56-5c12-4dde-8490-1644a989e085 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:30.280030 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a1f56-5c12-4dde-8490-1644a989e085 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:30.280308 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a1f56-5c12-4dde-8490-1644a989e085 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:30.281066 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a1f56-5c12-4dde-8490-1644a989e085 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:30.282469 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a1f56-5c12-4dde-8490-1644a989e085 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:30.292728 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:30.313588 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f5d7d-315b-4a6e-bbee-71c628fd87e7 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:30.326762 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:30.346648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528f97e-a74a-4a01-8504-5f12880ed1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:30.347546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528f97e-a74a-4a01-8504-5f12880ed1fb None admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:30.347826 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528f97e-a74a-4a01-8504-5f12880ed1fb None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:30.348100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528f97e-a74a-4a01-8504-5f12880ed1fb None admin] PATH_INFO: `/v3/domains/default/users/5aca81affd5542cf9480c6e76c8c5478/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:30.350660 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a528f97e-a74a-4a01-8504-5f12880ed1fb None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=5aca81affd5542cf9480c6e76c8c5478, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:30.352119 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a528f97e-a74a-4a01-8504-5f12880ed1fb None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:30.365238 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 22/42] 10.222.0.41 () {58 vars in 1361 bytes} [Fri Jun 3 16:01:30 2022] PUT /identity/v3/domains/default/users/5aca81affd5542cf9480c6e76c8c5478/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:01:30.366441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c13991e-0651-4fa6-b204-3a3a2ce8f6c9 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:30.366727 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c13991e-0651-4fa6-b204-3a3a2ce8f6c9 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:30.367006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c13991e-0651-4fa6-b204-3a3a2ce8f6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:30.397466 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-8c13991e-0651-4fa6-b204-3a3a2ce8f6c9 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:30.399037 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-8c13991e-0651-4fa6-b204-3a3a2ce8f6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:30.402247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 21/43] 10.222.0.41 () {60 vars in 1131 bytes} [Fri Jun 3 16:01:30 2022] POST /identity/v3/roles => generated 223 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:01:30.481723 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 23/44] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:01:30.505545 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a1f56-5c12-4dde-8490-1644a989e085 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:30.622900 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a1f56-5c12-4dde-8490-1644a989e085 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:30.659588 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe3414b-00d8-433c-aa8f-cee219698c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:30.660392 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3414b-00d8-433c-aa8f-cee219698c83 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:30.660704 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3414b-00d8-433c-aa8f-cee219698c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:30.660987 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3414b-00d8-433c-aa8f-cee219698c83 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:30.662319 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe3414b-00d8-433c-aa8f-cee219698c83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:30.663895 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe3414b-00d8-433c-aa8f-cee219698c83 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:30.665822 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 22/45] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:01:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:30.671207 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c13991e-0651-4fa6-b204-3a3a2ce8f6c9 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:30.773662 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c13991e-0651-4fa6-b204-3a3a2ce8f6c9 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:30.775957 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:30.808498 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a63d0d5-8cec-405d-a6fa-60bc1147f3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:30.809161 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63d0d5-8cec-405d-a6fa-60bc1147f3db None admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:30.809450 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63d0d5-8cec-405d-a6fa-60bc1147f3db None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:30.809731 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63d0d5-8cec-405d-a6fa-60bc1147f3db None admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:30.810452 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a63d0d5-8cec-405d-a6fa-60bc1147f3db None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:30.811933 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a63d0d5-8cec-405d-a6fa-60bc1147f3db None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:30.838512 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 24/46] 10.222.0.41 () {60 vars in 1141 bytes} [Fri Jun 3 16:01:30 2022] POST /identity/v3/projects => generated 312 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:01:32.656013 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d318a55-8636-40f7-9af5-eb7f68f66230 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:32.656509 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d318a55-8636-40f7-9af5-eb7f68f66230 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:32.656614 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d318a55-8636-40f7-9af5-eb7f68f66230 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:32.658072 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 23/47] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:01:32.663670 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101be446-c1e9-4ba3-ad58-d8afb2609a17 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:32.664057 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101be446-c1e9-4ba3-ad58-d8afb2609a17 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:32.664236 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101be446-c1e9-4ba3-ad58-d8afb2609a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:32.680868 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-101be446-c1e9-4ba3-ad58-d8afb2609a17 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:32.682052 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-101be446-c1e9-4ba3-ad58-d8afb2609a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:32.749335 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 25/48] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:01:32.773086 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d318a55-8636-40f7-9af5-eb7f68f66230 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:32.837822 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:32.875500 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d318a55-8636-40f7-9af5-eb7f68f66230 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:32.910205 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be1272d-3e4a-471b-9a0f-f7765fc68145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:32.910907 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1272d-3e4a-471b-9a0f-f7765fc68145 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:32.911193 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1272d-3e4a-471b-9a0f-f7765fc68145 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:32.911577 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1272d-3e4a-471b-9a0f-f7765fc68145 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:32.921345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be1272d-3e4a-471b-9a0f-f7765fc68145 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:01:32.928840 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-2be1272d-3e4a-471b-9a0f-f7765fc68145 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:01:32.928840 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:01:32.928840 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:01:32.928840 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:01:32.928840 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:01:32.928840 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:01:32.928840 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:01:32.928840 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:01:32.928840 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:01:32.928840 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:01:32.928840 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:01:32.928840 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:01:32.928840 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:01:32.928840 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:01:32.928840 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:01:32.928840 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:01:32.929708 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:01:32.929708 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:01:32.929708 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:01:32.929708 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:01:32.929708 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:01:32.929708 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:01:32.929708 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:01:32.929708 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:01:32.929708 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:01:32.929708 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:01:32.929708 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:01:32.929708 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:01:32.929708 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:01:32.929708 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:01:32.929708 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:01:32.929708 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:01:32.930455 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:01:32.930455 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:01:32.930455 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:01:32.930455 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:01:32.930455 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:01:32.930455 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:01:32.930455 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:01:32.930455 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:01:32.930455 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:01:32.930455 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:01:32.930455 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:01:32.930455 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:01:32.930455 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 24/49] 10.222.0.41 () {56 vars in 1096 bytes} [Fri Jun 3 16:01:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:01:32.936876 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101be446-c1e9-4ba3-ad58-d8afb2609a17 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:33.022475 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:33.038694 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101be446-c1e9-4ba3-ad58-d8afb2609a17 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:33.066792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6445b-537e-4c1e-b0e3-1ef55d9047ed None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:33.067087 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6445b-537e-4c1e-b0e3-1ef55d9047ed None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:33.067414 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6445b-537e-4c1e-b0e3-1ef55d9047ed None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:33.070519 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 25/50] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 16:01:33.071461 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c251f-661d-4b3f-be98-67462a20e6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:33.072128 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c251f-661d-4b3f-be98-67462a20e6ce None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:33.072415 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c251f-661d-4b3f-be98-67462a20e6ce None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:33.072689 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c251f-661d-4b3f-be98-67462a20e6ce None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:33.073442 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293c251f-661d-4b3f-be98-67462a20e6ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:33.073751 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293c251f-661d-4b3f-be98-67462a20e6ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:33.075120 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293c251f-661d-4b3f-be98-67462a20e6ce None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:33.075477 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be34df-c421-4e2c-adc2-b982acc8873d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:33.075771 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be34df-c421-4e2c-adc2-b982acc8873d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:33.076054 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be34df-c421-4e2c-adc2-b982acc8873d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:33.083326 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 26/51] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:01:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:33.089039 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c251f-661d-4b3f-be98-67462a20e6ce None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:33.093196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b5be34df-c421-4e2c-adc2-b982acc8873d None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:33.094728 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b5be34df-c421-4e2c-adc2-b982acc8873d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:33.160406 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 26/52] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 16:01:33.182767 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be34df-c421-4e2c-adc2-b982acc8873d None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:33.185601 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c251f-661d-4b3f-be98-67462a20e6ce None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:33.219590 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48595acb-970f-4b53-83fe-24e1232d256f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:33.220461 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48595acb-970f-4b53-83fe-24e1232d256f None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:33.220745 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48595acb-970f-4b53-83fe-24e1232d256f None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:33.221024 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48595acb-970f-4b53-83fe-24e1232d256f None admin] PATH_INFO: `/v3/users/5aca81affd5542cf9480c6e76c8c5478` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:33.222216 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48595acb-970f-4b53-83fe-24e1232d256f None admin] RBAC: Authorizing `identity:get_user(user_id=5aca81affd5542cf9480c6e76c8c5478)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:33.223702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48595acb-970f-4b53-83fe-24e1232d256f None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:33.225355 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 27/53] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:01:33 2022] GET /identity/v3/users/5aca81affd5542cf9480c6e76c8c5478 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:01:33.231105 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48595acb-970f-4b53-83fe-24e1232d256f None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:33.239778 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:33.252202 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:33.284890 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be34df-c421-4e2c-adc2-b982acc8873d None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:33.317681 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235b04f-0dd0-478e-8a5c-1006ab463807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:33.318463 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235b04f-0dd0-478e-8a5c-1006ab463807 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:33.319329 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235b04f-0dd0-478e-8a5c-1006ab463807 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:33.319612 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235b04f-0dd0-478e-8a5c-1006ab463807 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:33.320781 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235b04f-0dd0-478e-8a5c-1006ab463807 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:33.322237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235b04f-0dd0-478e-8a5c-1006ab463807 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:33.323860 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 27/54] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:01:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:01:33.329366 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235b04f-0dd0-478e-8a5c-1006ab463807 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:33.334140 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48595acb-970f-4b53-83fe-24e1232d256f None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:33.368431 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc764c5a-deb2-486d-93b2-42309c3768fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:33.369226 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc764c5a-deb2-486d-93b2-42309c3768fd None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:33.369527 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc764c5a-deb2-486d-93b2-42309c3768fd None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:33.369811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc764c5a-deb2-486d-93b2-42309c3768fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:33.370989 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc764c5a-deb2-486d-93b2-42309c3768fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:33.372499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc764c5a-deb2-486d-93b2-42309c3768fd None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:33.374114 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 28/55] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:01:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:01:33.379646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc764c5a-deb2-486d-93b2-42309c3768fd None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:33.423459 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235b04f-0dd0-478e-8a5c-1006ab463807 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:33.448619 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:33.459754 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db80a0-dc3a-4a15-b712-831de5d710b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:33.460444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db80a0-dc3a-4a15-b712-831de5d710b5 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:33.460742 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db80a0-dc3a-4a15-b712-831de5d710b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:33.461028 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db80a0-dc3a-4a15-b712-831de5d710b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:33.461146 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:33.461741 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db80a0-dc3a-4a15-b712-831de5d710b5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:33.463568 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db80a0-dc3a-4a15-b712-831de5d710b5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:33.477084 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc764c5a-deb2-486d-93b2-42309c3768fd None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:33.491669 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 28/56] 10.222.0.41 () {60 vars in 1140 bytes} [Fri Jun 3 16:01:33 2022] POST /identity/v3/projects => generated 298 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:01:33.520304 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ceb4a5-51c0-473a-afcf-d693cffa93a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:33.520996 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ceb4a5-51c0-473a-afcf-d693cffa93a5 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:33.521289 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ceb4a5-51c0-473a-afcf-d693cffa93a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:33.521576 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ceb4a5-51c0-473a-afcf-d693cffa93a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:33.522481 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ceb4a5-51c0-473a-afcf-d693cffa93a5 None admin] RBAC: Adding query filter params (role.id=75936a608b8149099ff54e317f3a6740, scope.domain.id=default, user.id=5aca81affd5542cf9480c6e76c8c5478) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:33.522802 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ceb4a5-51c0-473a-afcf-d693cffa93a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:33.524351 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ceb4a5-51c0-473a-afcf-d693cffa93a5 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:33.533860 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 29/57] 10.222.0.41 () {56 vars in 1322 bytes} [Fri Jun 3 16:01:33 2022] GET /identity/v3/role_assignments?user.id=5aca81affd5542cf9480c6e76c8c5478&scope.domain.id=default&role.id=75936a608b8149099ff54e317f3a6740 => generated 530 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:01:35.739581 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367a524-45a2-49ad-b49a-5dfe75ad9a65 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:35.740070 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367a524-45a2-49ad-b49a-5dfe75ad9a65 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:35.740152 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367a524-45a2-49ad-b49a-5dfe75ad9a65 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:35.742053 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 29/58] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:01:35.747217 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7390f3-0f11-44fc-bf33-761fa47ea489 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:35.747660 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7390f3-0f11-44fc-bf33-761fa47ea489 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:35.747814 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7390f3-0f11-44fc-bf33-761fa47ea489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:35.764864 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-4d7390f3-0f11-44fc-bf33-761fa47ea489 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:35.766062 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-4d7390f3-0f11-44fc-bf33-761fa47ea489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:35.830794 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 30/59] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:01:35.854575 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a367a524-45a2-49ad-b49a-5dfe75ad9a65 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:35.953075 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a367a524-45a2-49ad-b49a-5dfe75ad9a65 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:35.985700 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49394f4d-8e30-45a6-b098-da6b06a7d124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:35.986480 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49394f4d-8e30-45a6-b098-da6b06a7d124 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:35.986767 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49394f4d-8e30-45a6-b098-da6b06a7d124 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:35.987048 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49394f4d-8e30-45a6-b098-da6b06a7d124 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:35.988267 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49394f4d-8e30-45a6-b098-da6b06a7d124 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:35.989775 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49394f4d-8e30-45a6-b098-da6b06a7d124 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:35.992078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 30/60] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:01:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:36.001176 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7390f3-0f11-44fc-bf33-761fa47ea489 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:36.075012 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:36.095644 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7390f3-0f11-44fc-bf33-761fa47ea489 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:36.128294 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b5a47-c52b-4aa4-9789-4f30ca822da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:36.128951 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b5a47-c52b-4aa4-9789-4f30ca822da7 None admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:36.129236 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b5a47-c52b-4aa4-9789-4f30ca822da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:36.129514 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b5a47-c52b-4aa4-9789-4f30ca822da7 None admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:36.130222 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64b5a47-c52b-4aa4-9789-4f30ca822da7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:36.131749 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64b5a47-c52b-4aa4-9789-4f30ca822da7 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:36.156917 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 31/61] 10.222.0.41 () {60 vars in 1132 bytes} [Fri Jun 3 16:01:35 2022] POST /identity/v3/users => generated 275 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:01:39.411837 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c13e5-bcfe-41b5-95f0-78b183a97732 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:39.412870 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c13e5-bcfe-41b5-95f0-78b183a97732 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:39.413368 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c13e5-bcfe-41b5-95f0-78b183a97732 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:39.414949 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 31/62] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:01:39.420536 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610851f4-e82b-4b69-aba7-e2bb2cb71ca9 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:39.421369 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610851f4-e82b-4b69-aba7-e2bb2cb71ca9 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:39.421846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610851f4-e82b-4b69-aba7-e2bb2cb71ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:39.448569 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-610851f4-e82b-4b69-aba7-e2bb2cb71ca9 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:39.450088 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-610851f4-e82b-4b69-aba7-e2bb2cb71ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:39.566018 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 32/63] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:01:39.580784 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63bbe31-ee30-44de-97b3-955ce9c18dae None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:39.581078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63bbe31-ee30-44de-97b3-955ce9c18dae None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:39.581530 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63bbe31-ee30-44de-97b3-955ce9c18dae None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:39.583053 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 32/64] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:01:39.589145 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad0c7f-a55d-4777-8bc7-62bcf338d2fe None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:39.589442 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad0c7f-a55d-4777-8bc7-62bcf338d2fe None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:39.589728 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad0c7f-a55d-4777-8bc7-62bcf338d2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:39.607908 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63bbe31-ee30-44de-97b3-955ce9c18dae None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:39.620314 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-d0ad0c7f-a55d-4777-8bc7-62bcf338d2fe None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:39.621507 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-d0ad0c7f-a55d-4777-8bc7-62bcf338d2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:39.674284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:39.749864 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63bbe31-ee30-44de-97b3-955ce9c18dae None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:39.762320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:39.775908 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 33/65] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 16:01:39.799212 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad0c7f-a55d-4777-8bc7-62bcf338d2fe None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:39.808474 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44f2ee-7502-46c2-a7dc-f6e06d84d768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:39.809655 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44f2ee-7502-46c2-a7dc-f6e06d84d768 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:39.810142 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44f2ee-7502-46c2-a7dc-f6e06d84d768 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:39.810615 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44f2ee-7502-46c2-a7dc-f6e06d84d768 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:39.822591 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e44f2ee-7502-46c2-a7dc-f6e06d84d768 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:01:39.831095 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-4e44f2ee-7502-46c2-a7dc-f6e06d84d768 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 16:01:39.831095 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:01:39.831095 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:01:39.831095 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:01:39.831095 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:01:39.831095 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:01:39.831095 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:01:39.831095 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:01:39.831095 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:01:39.831095 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:01:39.831095 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:01:39.831095 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:01:39.831095 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:01:39.831095 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:01:39.831095 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:01:39.831095 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:01:39.831962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:01:39.831962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:01:39.831962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:01:39.831962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:01:39.831962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:01:39.831962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:01:39.831962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:01:39.831962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:01:39.831962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:01:39.831962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:01:39.831962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:01:39.831962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:01:39.831962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:01:39.831962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:01:39.831962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:01:39.831962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:01:39.832737 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:01:39.832737 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:01:39.832737 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:01:39.832737 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:01:39.832737 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:01:39.832737 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:01:39.832737 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:01:39.832737 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:01:39.832737 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:01:39.832737 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:01:39.832737 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 16:01:39.832737 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:01:39.836137 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 33/66] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:01:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:01:39.844671 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44f2ee-7502-46c2-a7dc-f6e06d84d768 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:39.912442 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad0c7f-a55d-4777-8bc7-62bcf338d2fe None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:39.950301 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab27139-4985-44fa-bb37-c3b98ca0b61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:39.951415 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab27139-4985-44fa-bb37-c3b98ca0b61f None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:39.951847 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab27139-4985-44fa-bb37-c3b98ca0b61f None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:39.952270 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab27139-4985-44fa-bb37-c3b98ca0b61f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:39.963923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab27139-4985-44fa-bb37-c3b98ca0b61f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:01:39.971933 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-6ab27139-4985-44fa-bb37-c3b98ca0b61f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:01:39.971933 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:01:39.971933 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:01:39.971933 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:01:39.971933 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:01:39.971933 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:01:39.971933 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:01:39.971933 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:01:39.971933 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:01:39.971933 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:01:39.971933 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:01:39.971933 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:01:39.971933 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:01:39.971933 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:01:39.971933 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:01:39.971933 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:01:39.972909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:01:39.972909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:01:39.972909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:01:39.972909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:01:39.972909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:01:39.972909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:01:39.972909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:01:39.972909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:01:39.972909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:01:39.972909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:01:39.972909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:01:39.972909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:01:39.972909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:01:39.972909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:01:39.972909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:01:39.972909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:01:39.973809 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:01:39.973809 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:01:39.973809 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:01:39.973809 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:01:39.973809 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:01:39.973809 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:01:39.973809 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:01:39.973809 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:01:39.973809 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:01:39.973809 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:01:39.973809 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:01:39.973809 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:01:39.976227 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 34/67] 10.222.0.41 () {56 vars in 1096 bytes} [Fri Jun 3 16:01:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:01:39.977733 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c59945-65e9-49cc-96fa-655eda86edc8 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:39.978216 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c59945-65e9-49cc-96fa-655eda86edc8 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:39.978699 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c59945-65e9-49cc-96fa-655eda86edc8 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:39.980205 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 35/68] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:01:39.982571 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c59945-65e9-49cc-96fa-655eda86edc8 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:40.048445 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:40.054848 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44f2ee-7502-46c2-a7dc-f6e06d84d768 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:40.078001 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:40.096796 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6f4ca6-14b5-489d-87ee-99855167b6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:40.097470 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f4ca6-14b5-489d-87ee-99855167b6a4 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:40.097761 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f4ca6-14b5-489d-87ee-99855167b6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:40.098046 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f4ca6-14b5-489d-87ee-99855167b6a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:40.098693 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c59945-65e9-49cc-96fa-655eda86edc8 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:40.098809 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6f4ca6-14b5-489d-87ee-99855167b6a4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:40.099329 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6f4ca6-14b5-489d-87ee-99855167b6a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:40.101301 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6f4ca6-14b5-489d-87ee-99855167b6a4 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:40.109843 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 34/69] 10.222.0.41 () {56 vars in 1101 bytes} [Fri Jun 3 16:01:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:40.111210 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2e82a-3861-42f3-9268-7d3083660511 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:40.111611 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2e82a-3861-42f3-9268-7d3083660511 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:40.111897 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2e82a-3861-42f3-9268-7d3083660511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:40.128407 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-f0e2e82a-3861-42f3-9268-7d3083660511 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:40.129610 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f0e2e82a-3861-42f3-9268-7d3083660511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:40.138402 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c03736d-0d6f-4a85-81d2-5a97cd840040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:40.139079 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03736d-0d6f-4a85-81d2-5a97cd840040 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:40.139392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03736d-0d6f-4a85-81d2-5a97cd840040 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:40.139672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03736d-0d6f-4a85-81d2-5a97cd840040 None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:40.140471 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c03736d-0d6f-4a85-81d2-5a97cd840040 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:40.140950 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c03736d-0d6f-4a85-81d2-5a97cd840040 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:40.142475 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c03736d-0d6f-4a85-81d2-5a97cd840040 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:40.153239 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 36/70] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:01:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:01:40.155035 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23208d8b-7a1b-45ed-8d06-abc97296fe6f None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:40.155345 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23208d8b-7a1b-45ed-8d06-abc97296fe6f None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:40.155629 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23208d8b-7a1b-45ed-8d06-abc97296fe6f None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:40.156844 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 37/71] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:01:40.157537 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23208d8b-7a1b-45ed-8d06-abc97296fe6f None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:40.207771 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 35/72] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:01:40.208505 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e2e82a-3861-42f3-9268-7d3083660511 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:40.271282 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23208d8b-7a1b-45ed-8d06-abc97296fe6f None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:40.290261 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:40.308474 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facbc244-5145-4025-832e-f1e351eb0175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:40.309352 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbc244-5145-4025-832e-f1e351eb0175 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:40.310003 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbc244-5145-4025-832e-f1e351eb0175 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:40.310174 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbc244-5145-4025-832e-f1e351eb0175 None admin] PATH_INFO: `/v3/users/f6a0ad4d1a084c0db8eb52bd4ed3a69a` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:40.313678 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:40.320028 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e2e82a-3861-42f3-9268-7d3083660511 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:40.325453 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facbc244-5145-4025-832e-f1e351eb0175 None admin] RBAC: Authorizing `identity:get_user(user_id=f6a0ad4d1a084c0db8eb52bd4ed3a69a)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:40.326895 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facbc244-5145-4025-832e-f1e351eb0175 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:40.328670 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 38/73] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:01:40 2022] GET /identity/v3/users/f6a0ad4d1a084c0db8eb52bd4ed3a69a => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:40.330029 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f289a-3680-4fc6-8b1a-f4d018ecbd70 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:40.330324 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f289a-3680-4fc6-8b1a-f4d018ecbd70 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:40.330614 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f289a-3680-4fc6-8b1a-f4d018ecbd70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:40.346938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-b72f289a-3680-4fc6-8b1a-f4d018ecbd70 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:40.348166 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-b72f289a-3680-4fc6-8b1a-f4d018ecbd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:40.355076 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a065b-0a93-413d-adb5-3ac70eadc465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:40.355984 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a065b-0a93-413d-adb5-3ac70eadc465 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:40.356274 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a065b-0a93-413d-adb5-3ac70eadc465 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:40.356553 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a065b-0a93-413d-adb5-3ac70eadc465 None admin] PATH_INFO: `/v3/users/5aca81affd5542cf9480c6e76c8c5478` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:40.357766 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a065b-0a93-413d-adb5-3ac70eadc465 None admin] RBAC: Authorizing `identity:get_user(user_id=5aca81affd5542cf9480c6e76c8c5478)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:40.359210 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a065b-0a93-413d-adb5-3ac70eadc465 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:40.360851 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 36/74] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:01:40 2022] GET /identity/v3/users/5aca81affd5542cf9480c6e76c8c5478 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:40.361600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a065b-0a93-413d-adb5-3ac70eadc465 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:40.415426 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 39/75] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:01:40.416727 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c3853-fb71-4fe5-adb8-8e8aa7ad3081 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:40.417021 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c3853-fb71-4fe5-adb8-8e8aa7ad3081 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:40.417302 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c3853-fb71-4fe5-adb8-8e8aa7ad3081 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:40.418558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 40/76] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:01:40.423224 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23c3853-fb71-4fe5-adb8-8e8aa7ad3081 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:40.469085 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a065b-0a93-413d-adb5-3ac70eadc465 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:40.501310 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d8b07-2efa-461c-90ae-303c63c3b145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:40.502120 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d8b07-2efa-461c-90ae-303c63c3b145 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:40.502404 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d8b07-2efa-461c-90ae-303c63c3b145 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:40.502682 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d8b07-2efa-461c-90ae-303c63c3b145 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:40.513164 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9d8b07-2efa-461c-90ae-303c63c3b145 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:01:40.521208 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-1b9d8b07-2efa-461c-90ae-303c63c3b145 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 16:01:40.521208 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:01:40.521208 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:01:40.521208 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:01:40.521208 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:01:40.521208 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:01:40.521208 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:01:40.521208 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:01:40.521208 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:01:40.521208 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:01:40.521208 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:01:40.521208 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:01:40.521208 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:01:40.521208 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:01:40.521208 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:01:40.521208 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:01:40.522625 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:01:40.522625 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:01:40.522625 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:01:40.522625 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:01:40.522625 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:01:40.522625 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:01:40.522625 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:01:40.522625 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:01:40.522625 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:01:40.522625 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:01:40.522625 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:01:40.522625 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:01:40.522625 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:01:40.522625 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:01:40.522625 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:01:40.522625 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:01:40.523406 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:01:40.523406 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:01:40.523406 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:01:40.523406 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:01:40.523406 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:01:40.523406 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:01:40.523406 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:01:40.523406 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:01:40.523406 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:01:40.523406 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:01:40.523406 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 16:01:40.523406 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:01:40.524243 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23c3853-fb71-4fe5-adb8-8e8aa7ad3081 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:40.524316 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 37/77] 10.222.0.41 () {56 vars in 1114 bytes} [Fri Jun 3 16:01:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:01:40.524316 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d8b07-2efa-461c-90ae-303c63c3b145 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:40.532923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:40.555556 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:40.561832 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079f764-d544-4c7f-8a92-e337c0db0235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:40.562516 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079f764-d544-4c7f-8a92-e337c0db0235 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:40.562807 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079f764-d544-4c7f-8a92-e337c0db0235 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:40.563085 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079f764-d544-4c7f-8a92-e337c0db0235 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:40.628148 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d8b07-2efa-461c-90ae-303c63c3b145 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:40.628985 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079f764-d544-4c7f-8a92-e337c0db0235 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:40.630483 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079f764-d544-4c7f-8a92-e337c0db0235 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:40.663215 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418707af-9ed9-478b-b7bc-0797b4ad17e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:40.663878 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418707af-9ed9-478b-b7bc-0797b4ad17e1 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:40.664165 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418707af-9ed9-478b-b7bc-0797b4ad17e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:40.664443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418707af-9ed9-478b-b7bc-0797b4ad17e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:40.730813 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418707af-9ed9-478b-b7bc-0797b4ad17e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:40.732339 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418707af-9ed9-478b-b7bc-0797b4ad17e1 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:40.732796 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 41/78] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:01:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 16:01:40.738526 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137337b-056c-417e-8a3e-aadbd8826c1b None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:40.738843 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137337b-056c-417e-8a3e-aadbd8826c1b None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:40.739124 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137337b-056c-417e-8a3e-aadbd8826c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:40.746190 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:40.757041 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-f137337b-056c-417e-8a3e-aadbd8826c1b None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:40.757685 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:40.758218 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-f137337b-056c-417e-8a3e-aadbd8826c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:40.824235 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 42/79] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:01:40.825053 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f137337b-056c-417e-8a3e-aadbd8826c1b None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:40.835765 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 38/80] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:01:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 16:01:40.840947 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418707af-9ed9-478b-b7bc-0797b4ad17e1 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:40.921766 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f137337b-056c-417e-8a3e-aadbd8826c1b None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:40.938691 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418707af-9ed9-478b-b7bc-0797b4ad17e1 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:40.954951 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac749c-e5c7-4531-9918-56fcf6d3b749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:40.955778 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac749c-e5c7-4531-9918-56fcf6d3b749 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:40.956064 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac749c-e5c7-4531-9918-56fcf6d3b749 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:40.956340 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac749c-e5c7-4531-9918-56fcf6d3b749 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:40.958155 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:40.962371 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:40.968305 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ac749c-e5c7-4531-9918-56fcf6d3b749 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:01:40.973378 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e48965-92cb-4f23-b074-13446ef6dce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:40.974050 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e48965-92cb-4f23-b074-13446ef6dce9 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:40.974333 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e48965-92cb-4f23-b074-13446ef6dce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:40.975141 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e48965-92cb-4f23-b074-13446ef6dce9 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:40.975959 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e48965-92cb-4f23-b074-13446ef6dce9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:40.976270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e48965-92cb-4f23-b074-13446ef6dce9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:40.976785 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-69ac749c-e5c7-4531-9918-56fcf6d3b749 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 16:01:40.976785 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:01:40.976785 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:01:40.976785 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:01:40.976785 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:01:40.976785 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:01:40.976785 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:01:40.976785 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:01:40.976785 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:01:40.976785 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:01:40.976785 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:01:40.976785 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:01:40.976785 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:01:40.976785 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:01:40.976785 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:01:40.976785 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:01:40.977716 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:01:40.977716 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:01:40.977716 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:01:40.977716 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:01:40.977716 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:01:40.977716 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:01:40.977716 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:01:40.977716 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:01:40.977716 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:01:40.977716 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:01:40.977716 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:01:40.977716 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:01:40.977716 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:01:40.977716 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:01:40.977716 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:01:40.977716 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:01:40.983558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:01:40.983558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:01:40.983558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:01:40.983558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:01:40.983558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:01:40.983558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:01:40.983558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:01:40.983558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:01:40.983558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:01:40.983558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:01:40.983558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 16:01:40.983558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:01:40.984127 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e48965-92cb-4f23-b074-13446ef6dce9 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:40.984196 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 43/81] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:01:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:01:40.984196 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac749c-e5c7-4531-9918-56fcf6d3b749 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:40.994985 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 39/82] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:01:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:40.995885 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e48965-92cb-4f23-b074-13446ef6dce9 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:41.081518 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac749c-e5c7-4531-9918-56fcf6d3b749 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:41.092767 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e48965-92cb-4f23-b074-13446ef6dce9 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:41.115010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cada72-0646-4585-971f-09aafd73af07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:41.115910 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cada72-0646-4585-971f-09aafd73af07 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:41.116196 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cada72-0646-4585-971f-09aafd73af07 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:41.116472 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cada72-0646-4585-971f-09aafd73af07 None admin] PATH_INFO: `/v3/projects/1d2895687ae2471682919de9b8fa4afb` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:41.117661 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cada72-0646-4585-971f-09aafd73af07 None admin] RBAC: Authorizing `identity:get_project(project_id=1d2895687ae2471682919de9b8fa4afb)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:41.119113 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cada72-0646-4585-971f-09aafd73af07 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:41.120847 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 44/83] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:01:40 2022] GET /identity/v3/projects/1d2895687ae2471682919de9b8fa4afb => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:41.121636 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cada72-0646-4585-971f-09aafd73af07 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:41.126393 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ddf645-a13c-4380-a7be-5ff10218e151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:41.127269 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ddf645-a13c-4380-a7be-5ff10218e151 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:41.127600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ddf645-a13c-4380-a7be-5ff10218e151 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:41.127882 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ddf645-a13c-4380-a7be-5ff10218e151 None admin] PATH_INFO: `/v3/projects/1d2895687ae2471682919de9b8fa4afb` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:41.129086 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ddf645-a13c-4380-a7be-5ff10218e151 None admin] RBAC: Authorizing `identity:get_project(project_id=1d2895687ae2471682919de9b8fa4afb)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:41.130538 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ddf645-a13c-4380-a7be-5ff10218e151 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:41.132773 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 40/84] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:01:40 2022] GET /identity/v3/projects/1d2895687ae2471682919de9b8fa4afb => generated 298 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:41.133527 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ddf645-a13c-4380-a7be-5ff10218e151 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:41.185050 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:41.186919 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:41.224322 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cada72-0646-4585-971f-09aafd73af07 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:41.232422 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ddf645-a13c-4380-a7be-5ff10218e151 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:41.261621 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eca75f-51a6-4aa9-b1c8-08c2e5b37389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:41.262426 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eca75f-51a6-4aa9-b1c8-08c2e5b37389 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:41.262717 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eca75f-51a6-4aa9-b1c8-08c2e5b37389 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:41.263002 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eca75f-51a6-4aa9-b1c8-08c2e5b37389 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:41.265960 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eca75f-51a6-4aa9-b1c8-08c2e5b37389 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:41.266059 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed39e00b-6d99-4dfa-817d-2d0f0fc5a6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:41.266638 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39e00b-6d99-4dfa-817d-2d0f0fc5a6bd None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:41.266925 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39e00b-6d99-4dfa-817d-2d0f0fc5a6bd None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:41.267207 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39e00b-6d99-4dfa-817d-2d0f0fc5a6bd None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:41.267439 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eca75f-51a6-4aa9-b1c8-08c2e5b37389 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:41.268002 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed39e00b-6d99-4dfa-817d-2d0f0fc5a6bd None admin] RBAC: Adding query filter params (name=member) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:41.268312 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed39e00b-6d99-4dfa-817d-2d0f0fc5a6bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:41.269064 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 45/85] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:01:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:41.269670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed39e00b-6d99-4dfa-817d-2d0f0fc5a6bd None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:41.269835 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eca75f-51a6-4aa9-b1c8-08c2e5b37389 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:41.278790 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 41/86] 10.222.0.41 () {56 vars in 1101 bytes} [Fri Jun 3 16:01:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:41.279877 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed39e00b-6d99-4dfa-817d-2d0f0fc5a6bd None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:41.372845 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eca75f-51a6-4aa9-b1c8-08c2e5b37389 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:41.376217 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed39e00b-6d99-4dfa-817d-2d0f0fc5a6bd None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:41.391354 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:41.400213 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:41.407175 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027f6fa-2d05-4042-ad4c-34a8c4ca4b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:41.408077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027f6fa-2d05-4042-ad4c-34a8c4ca4b78 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:41.408361 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027f6fa-2d05-4042-ad4c-34a8c4ca4b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:41.408637 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027f6fa-2d05-4042-ad4c-34a8c4ca4b78 None admin] PATH_INFO: `/v3/users/f6a0ad4d1a084c0db8eb52bd4ed3a69a` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:41.408924 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d7d007-0b0a-45a4-a67d-e902505b9ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:41.409572 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7d007-0b0a-45a4-a67d-e902505b9ecc None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:41.409821 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5027f6fa-2d05-4042-ad4c-34a8c4ca4b78 None admin] RBAC: Authorizing `identity:get_user(user_id=f6a0ad4d1a084c0db8eb52bd4ed3a69a)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:41.409908 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7d007-0b0a-45a4-a67d-e902505b9ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:41.410125 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7d007-0b0a-45a4-a67d-e902505b9ecc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:41.410960 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d7d007-0b0a-45a4-a67d-e902505b9ecc None admin] RBAC: Adding query filter params (role.id=a8e4174e3572497f9284c709a119dcff, scope.project.id=1d2895687ae2471682919de9b8fa4afb, user.id=f6a0ad4d1a084c0db8eb52bd4ed3a69a) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:41.411259 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5027f6fa-2d05-4042-ad4c-34a8c4ca4b78 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:41.411385 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d7d007-0b0a-45a4-a67d-e902505b9ecc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:41.413232 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d7d007-0b0a-45a4-a67d-e902505b9ecc None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:41.413595 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 46/87] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:01:41 2022] GET /identity/v3/users/f6a0ad4d1a084c0db8eb52bd4ed3a69a => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:41.414193 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027f6fa-2d05-4042-ad4c-34a8c4ca4b78 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:41.419710 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 42/88] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:01:41 2022] GET /identity/v3/role_assignments?user.id=f6a0ad4d1a084c0db8eb52bd4ed3a69a&scope.project.id=1d2895687ae2471682919de9b8fa4afb&role.id=a8e4174e3572497f9284c709a119dcff => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:41.421456 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d7d007-0b0a-45a4-a67d-e902505b9ecc None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:41.530302 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d7d007-0b0a-45a4-a67d-e902505b9ecc None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:41.538881 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027f6fa-2d05-4042-ad4c-34a8c4ca4b78 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:41.566055 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a867d67d-9ced-42b8-82b8-481d08fdb7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:41.566790 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867d67d-9ced-42b8-82b8-481d08fdb7f6 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:41.567028 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867d67d-9ced-42b8-82b8-481d08fdb7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:41.567329 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867d67d-9ced-42b8-82b8-481d08fdb7f6 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:41.568081 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a867d67d-9ced-42b8-82b8-481d08fdb7f6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:41.569789 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a867d67d-9ced-42b8-82b8-481d08fdb7f6 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:41.576060 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950839bb-dbf9-4249-8795-977209fffe69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:41.577018 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950839bb-dbf9-4249-8795-977209fffe69 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:41.577479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950839bb-dbf9-4249-8795-977209fffe69 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:41.577930 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950839bb-dbf9-4249-8795-977209fffe69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:41.579125 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950839bb-dbf9-4249-8795-977209fffe69 None admin] RBAC: Adding query filter params (role.id=75936a608b8149099ff54e317f3a6740, scope.project.id=1d2895687ae2471682919de9b8fa4afb, user.id=5aca81affd5542cf9480c6e76c8c5478) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:41.579536 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950839bb-dbf9-4249-8795-977209fffe69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:41.581015 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950839bb-dbf9-4249-8795-977209fffe69 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:41.588576 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 47/89] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:01:41 2022] GET /identity/v3/role_assignments?user.id=5aca81affd5542cf9480c6e76c8c5478&scope.project.id=1d2895687ae2471682919de9b8fa4afb&role.id=75936a608b8149099ff54e317f3a6740 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:41.589421 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950839bb-dbf9-4249-8795-977209fffe69 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:41.614498 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 43/90] 10.222.0.41 () {60 vars in 1132 bytes} [Fri Jun 3 16:01:41 2022] POST /identity/v3/users => generated 289 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:01:41.616513 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a867d67d-9ced-42b8-82b8-481d08fdb7f6 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:41.688569 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:41.733967 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:41.811036 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a867d67d-9ced-42b8-82b8-481d08fdb7f6 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:41.851735 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f76f49-8e70-4405-b4f1-3170e2cb9d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:41.852430 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950839bb-dbf9-4249-8795-977209fffe69 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:41.852542 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f76f49-8e70-4405-b4f1-3170e2cb9d1e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:41.852729 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f76f49-8e70-4405-b4f1-3170e2cb9d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:41.853014 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f76f49-8e70-4405-b4f1-3170e2cb9d1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:41.908844 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b890a6-de48-4c3b-8fb8-5f6d8947b471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:41.910182 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b890a6-de48-4c3b-8fb8-5f6d8947b471 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:41.910476 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b890a6-de48-4c3b-8fb8-5f6d8947b471 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:41.910758 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b890a6-de48-4c3b-8fb8-5f6d8947b471 None admin] PATH_INFO: `/v3/users/f6a0ad4d1a084c0db8eb52bd4ed3a69a` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:41.912063 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b890a6-de48-4c3b-8fb8-5f6d8947b471 None admin] RBAC: Authorizing `identity:get_user(user_id=f6a0ad4d1a084c0db8eb52bd4ed3a69a)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:41.913962 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b890a6-de48-4c3b-8fb8-5f6d8947b471 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:41.916069 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 48/91] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:01:41 2022] GET /identity/v3/users/f6a0ad4d1a084c0db8eb52bd4ed3a69a => generated 275 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:41.922120 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b890a6-de48-4c3b-8fb8-5f6d8947b471 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:41.934816 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f76f49-8e70-4405-b4f1-3170e2cb9d1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:41.936464 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f76f49-8e70-4405-b4f1-3170e2cb9d1e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:41.984728 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:42.051082 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b890a6-de48-4c3b-8fb8-5f6d8947b471 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:42.066168 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 44/92] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:01:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 16:01:42.072351 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f76f49-8e70-4405-b4f1-3170e2cb9d1e None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:42.096071 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:42.104856 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5703475-e7f1-4c2f-90ad-c31170e3a5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:42.106259 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5703475-e7f1-4c2f-90ad-c31170e3a5a3 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:42.106690 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5703475-e7f1-4c2f-90ad-c31170e3a5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:42.107110 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5703475-e7f1-4c2f-90ad-c31170e3a5a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:42.190025 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5703475-e7f1-4c2f-90ad-c31170e3a5a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:42.192571 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5703475-e7f1-4c2f-90ad-c31170e3a5a3 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:42.197825 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f76f49-8e70-4405-b4f1-3170e2cb9d1e None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:42.237017 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b803c-19c0-44d2-a78c-95691b0a2fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:42.238181 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b803c-19c0-44d2-a78c-95691b0a2fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:42.238635 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b803c-19c0-44d2-a78c-95691b0a2fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:42.239105 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b803c-19c0-44d2-a78c-95691b0a2fa9 None admin] PATH_INFO: `/v3/projects/1d2895687ae2471682919de9b8fa4afb` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:42.241048 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428b803c-19c0-44d2-a78c-95691b0a2fa9 None admin] RBAC: Authorizing `identity:get_project(project_id=1d2895687ae2471682919de9b8fa4afb)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:42.242668 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428b803c-19c0-44d2-a78c-95691b0a2fa9 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:42.245129 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 45/93] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:01:42 2022] GET /identity/v3/projects/1d2895687ae2471682919de9b8fa4afb => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:42.258873 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b803c-19c0-44d2-a78c-95691b0a2fa9 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:42.264600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:42.329044 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 49/94] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:01:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 16:01:42.334659 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5703475-e7f1-4c2f-90ad-c31170e3a5a3 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:42.367702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:42.395934 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b803c-19c0-44d2-a78c-95691b0a2fa9 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:42.433666 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e7a5f5-9f9a-4fc0-b4ac-67efd8be7d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:42.434338 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7a5f5-9f9a-4fc0-b4ac-67efd8be7d75 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:42.434634 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7a5f5-9f9a-4fc0-b4ac-67efd8be7d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:42.434916 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7a5f5-9f9a-4fc0-b4ac-67efd8be7d75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:42.435813 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e7a5f5-9f9a-4fc0-b4ac-67efd8be7d75 None admin] RBAC: Adding query filter params (role.id=2248ac4d70c34e459b8942dec0d7656b, scope.project.id=1d2895687ae2471682919de9b8fa4afb, user.id=f6a0ad4d1a084c0db8eb52bd4ed3a69a) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:42.436132 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e7a5f5-9f9a-4fc0-b4ac-67efd8be7d75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:42.437684 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e7a5f5-9f9a-4fc0-b4ac-67efd8be7d75 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:42.444532 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 46/95] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:01:42 2022] GET /identity/v3/role_assignments?user.id=f6a0ad4d1a084c0db8eb52bd4ed3a69a&scope.project.id=1d2895687ae2471682919de9b8fa4afb&role.id=2248ac4d70c34e459b8942dec0d7656b => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:01:42.445468 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5703475-e7f1-4c2f-90ad-c31170e3a5a3 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:42.485230 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c470ac66-09b5-437d-9101-045e1a39a14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:42.486131 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470ac66-09b5-437d-9101-045e1a39a14f None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:42.486416 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470ac66-09b5-437d-9101-045e1a39a14f None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:42.486693 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470ac66-09b5-437d-9101-045e1a39a14f None admin] PATH_INFO: `/v3/projects/0be3fd66a8a74fb8861ddd3788b1f6ca` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:42.487988 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c470ac66-09b5-437d-9101-045e1a39a14f None admin] RBAC: Authorizing `identity:get_project(project_id=0be3fd66a8a74fb8861ddd3788b1f6ca)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:42.489469 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c470ac66-09b5-437d-9101-045e1a39a14f None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:42.492045 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 50/96] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:01:42 2022] GET /identity/v3/projects/0be3fd66a8a74fb8861ddd3788b1f6ca => generated 312 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:01:42.497783 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e7a5f5-9f9a-4fc0-b4ac-67efd8be7d75 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:42.592277 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:42.639096 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e7a5f5-9f9a-4fc0-b4ac-67efd8be7d75 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:42.686535 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153847f2-c18b-4383-864b-08007433f3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:42.687253 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153847f2-c18b-4383-864b-08007433f3ae None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:42.687593 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153847f2-c18b-4383-864b-08007433f3ae None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:42.687878 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153847f2-c18b-4383-864b-08007433f3ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:42.688727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153847f2-c18b-4383-864b-08007433f3ae None admin] RBAC: Adding query filter params (role.id=a8e4174e3572497f9284c709a119dcff, scope.project.id=0be3fd66a8a74fb8861ddd3788b1f6ca, user.id=f6a0ad4d1a084c0db8eb52bd4ed3a69a) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:42.689046 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153847f2-c18b-4383-864b-08007433f3ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:42.690702 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153847f2-c18b-4383-864b-08007433f3ae None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:42.697474 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 47/97] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:01:42 2022] GET /identity/v3/role_assignments?user.id=f6a0ad4d1a084c0db8eb52bd4ed3a69a&scope.project.id=0be3fd66a8a74fb8861ddd3788b1f6ca&role.id=a8e4174e3572497f9284c709a119dcff => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:45.121426 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4376ec-457d-4bbf-8dde-46f32bf4a9c2 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:45.122056 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4376ec-457d-4bbf-8dde-46f32bf4a9c2 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:45.122056 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4376ec-457d-4bbf-8dde-46f32bf4a9c2 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:45.123366 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 51/98] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:01:45.129064 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdca89-8802-4f1f-8e0c-0758b044a254 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:45.129507 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdca89-8802-4f1f-8e0c-0758b044a254 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:45.129692 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdca89-8802-4f1f-8e0c-0758b044a254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:45.156289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-4ffdca89-8802-4f1f-8e0c-0758b044a254 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:45.157511 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4ffdca89-8802-4f1f-8e0c-0758b044a254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:45.245962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 48/99] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:01:45.280926 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4376ec-457d-4bbf-8dde-46f32bf4a9c2 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:45.301982 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9507f82-4be5-4b8a-b9bc-ba11cd824dbf None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:45.302299 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9507f82-4be5-4b8a-b9bc-ba11cd824dbf None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:45.302571 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9507f82-4be5-4b8a-b9bc-ba11cd824dbf None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:45.304217 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 49/100] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:01:45.309894 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351565f-d0a7-4580-a67a-03f840f37b0c None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:45.310179 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351565f-d0a7-4580-a67a-03f840f37b0c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:45.310464 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351565f-d0a7-4580-a67a-03f840f37b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:45.328170 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-5351565f-d0a7-4580-a67a-03f840f37b0c None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:45.329370 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5351565f-d0a7-4580-a67a-03f840f37b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:45.411629 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:45.436806 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 50/101] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 16:01:45.450081 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:45.455045 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4376ec-457d-4bbf-8dde-46f32bf4a9c2 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:45.493285 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351565f-d0a7-4580-a67a-03f840f37b0c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:45.509690 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5157f-7af0-4d5f-be88-c07755c52347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:45.510502 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5157f-7af0-4d5f-be88-c07755c52347 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:45.510803 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5157f-7af0-4d5f-be88-c07755c52347 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:45.511066 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5157f-7af0-4d5f-be88-c07755c52347 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:45.522871 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5157f-7af0-4d5f-be88-c07755c52347 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:01:45.531423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-dab5157f-7af0-4d5f-be88-c07755c52347 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 16:01:45.531423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:01:45.531423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:01:45.531423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:01:45.531423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:01:45.531423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:01:45.531423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:01:45.531423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:01:45.531423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:01:45.531423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:01:45.531423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:01:45.531423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:01:45.531423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:01:45.531423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:01:45.531423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:01:45.531423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:01:45.532329 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:01:45.532329 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:01:45.532329 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:01:45.532329 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:01:45.532329 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:01:45.532329 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:01:45.532329 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:01:45.532329 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:01:45.532329 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:01:45.532329 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:01:45.532329 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:01:45.532329 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:01:45.532329 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:01:45.532329 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:01:45.532329 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:01:45.532329 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:01:45.533923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:01:45.533923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:01:45.533923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:01:45.533923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:01:45.533923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:01:45.533923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:01:45.533923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:01:45.533923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:01:45.533923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:01:45.533923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:01:45.533923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 16:01:45.533923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:01:45.533923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 52/102] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:01:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:01:45.537784 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0df6014-63ec-4947-9ef6-2017f4154f74 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:45.537989 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0df6014-63ec-4947-9ef6-2017f4154f74 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:45.538382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0df6014-63ec-4947-9ef6-2017f4154f74 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:45.539831 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 53/103] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:01:45.542637 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0df6014-63ec-4947-9ef6-2017f4154f74 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:45.639590 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351565f-d0a7-4580-a67a-03f840f37b0c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:45.678477 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc83b10-3389-4a51-9acd-b32f549435d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:45.679189 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc83b10-3389-4a51-9acd-b32f549435d8 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:45.679505 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc83b10-3389-4a51-9acd-b32f549435d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:45.679782 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc83b10-3389-4a51-9acd-b32f549435d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:45.688177 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0df6014-63ec-4947-9ef6-2017f4154f74 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:45.689846 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc83b10-3389-4a51-9acd-b32f549435d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:01:45.697398 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-1bc83b10-3389-4a51-9acd-b32f549435d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:01:45.697398 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:01:45.697398 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:01:45.697398 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:01:45.697398 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:01:45.697398 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:01:45.697398 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:01:45.697398 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:01:45.697398 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:01:45.697398 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:01:45.697398 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:01:45.697398 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:01:45.697398 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:01:45.697398 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:01:45.697398 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:01:45.697398 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:01:45.698229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:01:45.698229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:01:45.698229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:01:45.698229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:01:45.698229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:01:45.698229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:01:45.698229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:01:45.698229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:01:45.698229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:01:45.698229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:01:45.698229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:01:45.698229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:01:45.698229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:01:45.698229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:01:45.698229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:01:45.698229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:01:45.698977 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:01:45.698977 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:01:45.698977 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:01:45.698977 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:01:45.698977 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:01:45.698977 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:01:45.698977 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:01:45.698977 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:01:45.698977 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:01:45.698977 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:01:45.698977 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:01:45.698977 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:01:45.698977 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 51/104] 10.222.0.41 () {56 vars in 1096 bytes} [Fri Jun 3 16:01:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:01:45.700166 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511f94c-284c-407d-a154-b919c379139f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:45.700166 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511f94c-284c-407d-a154-b919c379139f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:45.700443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511f94c-284c-407d-a154-b919c379139f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:45.716132 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-a511f94c-284c-407d-a154-b919c379139f None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:45.717305 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a511f94c-284c-407d-a154-b919c379139f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:45.722401 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a1d08-f77a-4b09-b031-502526c22aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:45.723073 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a1d08-f77a-4b09-b031-502526c22aab None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:45.723393 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a1d08-f77a-4b09-b031-502526c22aab None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:45.723680 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a1d08-f77a-4b09-b031-502526c22aab None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:45.724951 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a1d08-f77a-4b09-b031-502526c22aab None admin] RBAC: Adding query filter params (name=member) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:45.725268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a1d08-f77a-4b09-b031-502526c22aab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:45.726656 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a1d08-f77a-4b09-b031-502526c22aab None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:45.735614 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 54/105] 10.222.0.41 () {56 vars in 1101 bytes} [Fri Jun 3 16:01:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:45.736844 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f28443-fb3d-47e4-bf10-a1ac7c640b84 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:45.737137 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f28443-fb3d-47e4-bf10-a1ac7c640b84 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:45.737419 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f28443-fb3d-47e4-bf10-a1ac7c640b84 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:45.738804 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 55/106] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:01:45.741816 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d0295-0e16-439a-b53c-3fe1a007b71b None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:45.742106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d0295-0e16-439a-b53c-3fe1a007b71b None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:45.742393 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d0295-0e16-439a-b53c-3fe1a007b71b None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:45.743676 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 56/107] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:01:45.747566 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c92ae-05e5-4d72-887a-52252e1401e7 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:45.747566 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c92ae-05e5-4d72-887a-52252e1401e7 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:45.747566 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c92ae-05e5-4d72-887a-52252e1401e7 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:45.747566 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 57/108] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:01:45.747566 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c92ae-05e5-4d72-887a-52252e1401e7 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:45.768214 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:45.771288 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:45.792212 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 52/109] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:01:45.792951 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a511f94c-284c-407d-a154-b919c379139f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:45.847240 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c92ae-05e5-4d72-887a-52252e1401e7 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:45.880607 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be45a3-a548-44a7-af1d-c257bde9f91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:45.881463 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be45a3-a548-44a7-af1d-c257bde9f91f None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:45.881760 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be45a3-a548-44a7-af1d-c257bde9f91f None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:45.882043 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be45a3-a548-44a7-af1d-c257bde9f91f None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:45.882791 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be45a3-a548-44a7-af1d-c257bde9f91f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:45.883105 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be45a3-a548-44a7-af1d-c257bde9f91f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:45.884524 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be45a3-a548-44a7-af1d-c257bde9f91f None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:45.889974 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a511f94c-284c-407d-a154-b919c379139f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:45.892458 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 58/110] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:01:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:45.892458 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c0877-6a31-43c5-b821-d382e96af1ed None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:45.892458 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c0877-6a31-43c5-b821-d382e96af1ed None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:45.892458 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c0877-6a31-43c5-b821-d382e96af1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:45.911148 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-ff2c0877-6a31-43c5-b821-d382e96af1ed None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:45.912900 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-ff2c0877-6a31-43c5-b821-d382e96af1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:45.923024 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de3576-1c7a-407a-86fd-73d786de3ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:45.923949 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de3576-1c7a-407a-86fd-73d786de3ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:45.924232 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de3576-1c7a-407a-86fd-73d786de3ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:45.924508 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de3576-1c7a-407a-86fd-73d786de3ca3 None admin] PATH_INFO: `/v3/users/f6a0ad4d1a084c0db8eb52bd4ed3a69a` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:45.925693 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de3576-1c7a-407a-86fd-73d786de3ca3 None admin] RBAC: Authorizing `identity:get_user(user_id=f6a0ad4d1a084c0db8eb52bd4ed3a69a)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:45.927137 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de3576-1c7a-407a-86fd-73d786de3ca3 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:45.929395 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 53/111] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:01:45 2022] GET /identity/v3/users/f6a0ad4d1a084c0db8eb52bd4ed3a69a => generated 275 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:45.930452 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7dd40-417c-4e58-b465-30888a3e8867 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:45.930740 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7dd40-417c-4e58-b465-30888a3e8867 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:45.931018 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7dd40-417c-4e58-b465-30888a3e8867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:45.946309 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c8c7dd40-417c-4e58-b465-30888a3e8867 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:45.947500 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c8c7dd40-417c-4e58-b465-30888a3e8867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:45.983483 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 59/112] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:01:45.984745 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40158a-63e7-4716-b40a-7d924c17ae3f None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:45.985036 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40158a-63e7-4716-b40a-7d924c17ae3f None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:45.985312 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40158a-63e7-4716-b40a-7d924c17ae3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:46.001061 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-cd40158a-63e7-4716-b40a-7d924c17ae3f None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:46.001385 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:46.002409 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-cd40158a-63e7-4716-b40a-7d924c17ae3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:46.010122 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:46.024702 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 54/113] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:01:46.025564 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7dd40-417c-4e58-b465-30888a3e8867 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:46.070482 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 60/114] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:01:46.071364 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd40158a-63e7-4716-b40a-7d924c17ae3f None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:46.126892 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7dd40-417c-4e58-b465-30888a3e8867 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:46.160687 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83204d49-da7b-45bc-8b36-b9fbfd9710ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:46.161688 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83204d49-da7b-45bc-8b36-b9fbfd9710ec None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:46.162572 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83204d49-da7b-45bc-8b36-b9fbfd9710ec None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:46.162877 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83204d49-da7b-45bc-8b36-b9fbfd9710ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:46.164136 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83204d49-da7b-45bc-8b36-b9fbfd9710ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:46.165680 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83204d49-da7b-45bc-8b36-b9fbfd9710ec None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:46.167745 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 55/115] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:01:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:46.168561 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83204d49-da7b-45bc-8b36-b9fbfd9710ec None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:46.181180 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd40158a-63e7-4716-b40a-7d924c17ae3f None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:46.214412 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb4fee-f4fa-4afc-9d43-43acec3c43b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:46.215283 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb4fee-f4fa-4afc-9d43-43acec3c43b8 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:46.215606 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb4fee-f4fa-4afc-9d43-43acec3c43b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:46.215889 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb4fee-f4fa-4afc-9d43-43acec3c43b8 None admin] PATH_INFO: `/v3/users/5aca81affd5542cf9480c6e76c8c5478` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:46.217077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eb4fee-f4fa-4afc-9d43-43acec3c43b8 None admin] RBAC: Authorizing `identity:get_user(user_id=5aca81affd5542cf9480c6e76c8c5478)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:46.218514 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eb4fee-f4fa-4afc-9d43-43acec3c43b8 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:46.218946 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:46.220170 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 61/116] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:01:46 2022] GET /identity/v3/users/5aca81affd5542cf9480c6e76c8c5478 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:46.220978 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb4fee-f4fa-4afc-9d43-43acec3c43b8 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:46.234548 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:46.275205 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83204d49-da7b-45bc-8b36-b9fbfd9710ec None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:46.308467 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f65862-da31-4ffc-a535-3fe3480a07e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:46.309328 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f65862-da31-4ffc-a535-3fe3480a07e1 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:46.309614 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f65862-da31-4ffc-a535-3fe3480a07e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:46.309892 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f65862-da31-4ffc-a535-3fe3480a07e1 None admin] PATH_INFO: `/v3/projects/1d2895687ae2471682919de9b8fa4afb` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:46.311074 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f65862-da31-4ffc-a535-3fe3480a07e1 None admin] RBAC: Authorizing `identity:get_project(project_id=1d2895687ae2471682919de9b8fa4afb)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:46.312578 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f65862-da31-4ffc-a535-3fe3480a07e1 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:46.314279 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 56/117] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:01:46 2022] GET /identity/v3/projects/1d2895687ae2471682919de9b8fa4afb => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:46.315091 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f65862-da31-4ffc-a535-3fe3480a07e1 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:46.320743 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb4fee-f4fa-4afc-9d43-43acec3c43b8 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:46.354283 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7782204c-127e-410e-abdd-8302a79a85ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:46.355075 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782204c-127e-410e-abdd-8302a79a85ec None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:46.355393 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782204c-127e-410e-abdd-8302a79a85ec None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:46.355677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782204c-127e-410e-abdd-8302a79a85ec None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:46.365525 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7782204c-127e-410e-abdd-8302a79a85ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:01:46.373141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-7782204c-127e-410e-abdd-8302a79a85ec None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 16:01:46.373141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:01:46.373141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:01:46.373141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:01:46.373141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:01:46.373141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:01:46.373141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:01:46.373141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:01:46.373141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:01:46.373141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:01:46.373141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:01:46.373141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:01:46.373141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:01:46.373141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:01:46.373141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:01:46.373141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:01:46.373958 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:01:46.373958 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:01:46.373958 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:01:46.373958 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:01:46.373958 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:01:46.373958 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:01:46.373958 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:01:46.373958 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:01:46.373958 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:01:46.373958 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:01:46.373958 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:01:46.373958 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:01:46.373958 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:01:46.373958 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:01:46.373958 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:01:46.373958 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:01:46.374754 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:01:46.374754 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:01:46.374754 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:01:46.374754 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:01:46.374754 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:01:46.374754 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:01:46.374754 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:01:46.374754 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:01:46.374754 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:01:46.374754 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:01:46.374754 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 16:01:46.374754 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:01:46.374754 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 62/118] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:01:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:01:46.375652 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7782204c-127e-410e-abdd-8302a79a85ec None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:46.417343 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f65862-da31-4ffc-a535-3fe3480a07e1 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:46.433008 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:46.446358 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:46.450635 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807eadfc-ba48-4fc9-80fb-5caae4a20294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:46.451464 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807eadfc-ba48-4fc9-80fb-5caae4a20294 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:46.451754 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807eadfc-ba48-4fc9-80fb-5caae4a20294 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:46.452033 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807eadfc-ba48-4fc9-80fb-5caae4a20294 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:46.462315 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807eadfc-ba48-4fc9-80fb-5caae4a20294 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:01:46.469949 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-807eadfc-ba48-4fc9-80fb-5caae4a20294 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 16:01:46.469949 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:01:46.469949 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:01:46.469949 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:01:46.469949 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:01:46.469949 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:01:46.469949 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:01:46.469949 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:01:46.469949 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:01:46.469949 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:01:46.469949 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:01:46.469949 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:01:46.469949 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:01:46.469949 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:01:46.469949 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:01:46.469949 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:01:46.470953 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:01:46.470953 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:01:46.470953 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:01:46.470953 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:01:46.470953 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:01:46.470953 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:01:46.470953 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:01:46.470953 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:01:46.470953 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:01:46.470953 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:01:46.470953 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:01:46.470953 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:01:46.470953 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:01:46.470953 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:01:46.470953 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:01:46.470953 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:01:46.471944 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:01:46.471944 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:01:46.471944 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:01:46.471944 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:01:46.471944 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:01:46.471944 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:01:46.471944 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:01:46.471944 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:01:46.471944 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:01:46.471944 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:01:46.471944 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 16:01:46.471944 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:01:46.471944 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 57/119] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:01:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:01:46.471944 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807eadfc-ba48-4fc9-80fb-5caae4a20294 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:46.481250 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7782204c-127e-410e-abdd-8302a79a85ec None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:46.514466 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b79010-f751-4991-b901-8dfe3c147867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:46.515383 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b79010-f751-4991-b901-8dfe3c147867 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:46.515676 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b79010-f751-4991-b901-8dfe3c147867 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:46.515961 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b79010-f751-4991-b901-8dfe3c147867 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:46.526607 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b79010-f751-4991-b901-8dfe3c147867 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:01:46.534968 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-75b79010-f751-4991-b901-8dfe3c147867 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 16:01:46.534968 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:01:46.534968 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:01:46.534968 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:01:46.534968 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:01:46.534968 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:01:46.534968 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:01:46.534968 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:01:46.534968 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:01:46.534968 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:01:46.534968 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:01:46.534968 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:01:46.534968 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:01:46.534968 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:01:46.534968 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:01:46.534968 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:01:46.536092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:01:46.536092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:01:46.536092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:01:46.536092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:01:46.536092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:01:46.536092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:01:46.536092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:01:46.536092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:01:46.536092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:01:46.536092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:01:46.536092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:01:46.536092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:01:46.536092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:01:46.536092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:01:46.536092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:01:46.536092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:01:46.536897 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:01:46.536897 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:01:46.536897 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:01:46.536897 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:01:46.536897 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:01:46.536897 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:01:46.536897 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:01:46.536897 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:01:46.536897 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:01:46.536897 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:01:46.536897 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 16:01:46.536897 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:01:46.536897 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 63/120] 10.222.0.41 () {56 vars in 1114 bytes} [Fri Jun 3 16:01:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:01:46.537510 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b79010-f751-4991-b901-8dfe3c147867 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:46.574020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807eadfc-ba48-4fc9-80fb-5caae4a20294 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:46.606330 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c770a-3606-4a14-931d-4163da64eda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:46.606975 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c770a-3606-4a14-931d-4163da64eda1 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:46.607261 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c770a-3606-4a14-931d-4163da64eda1 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:46.607572 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c770a-3606-4a14-931d-4163da64eda1 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:46.608276 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c770a-3606-4a14-931d-4163da64eda1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:46.609728 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c770a-3606-4a14-931d-4163da64eda1 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:46.633962 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b79010-f751-4991-b901-8dfe3c147867 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:46.635554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 58/121] 10.222.0.41 () {60 vars in 1141 bytes} [Fri Jun 3 16:01:46 2022] POST /identity/v3/projects => generated 302 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:01:46.636154 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c770a-3606-4a14-931d-4163da64eda1 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:46.670084 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:46.683261 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:46.689869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c315c01-e08f-4395-9d24-09263a77a671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:46.690776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c315c01-e08f-4395-9d24-09263a77a671 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:46.691069 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c315c01-e08f-4395-9d24-09263a77a671 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:46.691455 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c315c01-e08f-4395-9d24-09263a77a671 None admin] PATH_INFO: `/v3/projects/1d2895687ae2471682919de9b8fa4afb` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:46.692758 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c315c01-e08f-4395-9d24-09263a77a671 None admin] RBAC: Authorizing `identity:get_project(project_id=1d2895687ae2471682919de9b8fa4afb)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:46.694777 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c315c01-e08f-4395-9d24-09263a77a671 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:46.696678 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 64/122] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:01:46 2022] GET /identity/v3/projects/1d2895687ae2471682919de9b8fa4afb => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:46.701599 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c315c01-e08f-4395-9d24-09263a77a671 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:46.762969 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c770a-3606-4a14-931d-4163da64eda1 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:46.799812 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75399ad-7e18-46ee-aa32-8aba2c0f2515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:46.800880 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75399ad-7e18-46ee-aa32-8aba2c0f2515 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:46.801167 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75399ad-7e18-46ee-aa32-8aba2c0f2515 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:46.802028 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75399ad-7e18-46ee-aa32-8aba2c0f2515 None admin] PATH_INFO: `/v3/projects/1d2895687ae2471682919de9b8fa4afb/users/f6a0ad4d1a084c0db8eb52bd4ed3a69a/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:46.805150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75399ad-7e18-46ee-aa32-8aba2c0f2515 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d2895687ae2471682919de9b8fa4afb, user_id=f6a0ad4d1a084c0db8eb52bd4ed3a69a, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:46.807047 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75399ad-7e18-46ee-aa32-8aba2c0f2515 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:46.815317 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c315c01-e08f-4395-9d24-09263a77a671 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:46.825081 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 59/123] 10.222.0.41 () {58 vars in 1439 bytes} [Fri Jun 3 16:01:46 2022] PUT /identity/v3/projects/1d2895687ae2471682919de9b8fa4afb/users/f6a0ad4d1a084c0db8eb52bd4ed3a69a/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:01:46.825689 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75399ad-7e18-46ee-aa32-8aba2c0f2515 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:46.869701 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87fe367-07db-4b3e-b5e5-64524aba37ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:46.870995 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87fe367-07db-4b3e-b5e5-64524aba37ba None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:46.871331 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87fe367-07db-4b3e-b5e5-64524aba37ba None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:46.871607 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87fe367-07db-4b3e-b5e5-64524aba37ba None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:46.872389 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87fe367-07db-4b3e-b5e5-64524aba37ba None admin] RBAC: Adding query filter params (name=reader) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:46.872702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87fe367-07db-4b3e-b5e5-64524aba37ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:46.874148 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87fe367-07db-4b3e-b5e5-64524aba37ba None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:46.883913 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 65/124] 10.222.0.41 () {56 vars in 1101 bytes} [Fri Jun 3 16:01:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:46.884994 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87fe367-07db-4b3e-b5e5-64524aba37ba None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:46.958367 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:46.977615 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75399ad-7e18-46ee-aa32-8aba2c0f2515 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:47.015719 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:47.026029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeadeb5-5b74-4af2-9e36-254da6e095cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:47.026936 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87fe367-07db-4b3e-b5e5-64524aba37ba None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:47.027055 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeadeb5-5b74-4af2-9e36-254da6e095cb None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:47.027416 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeadeb5-5b74-4af2-9e36-254da6e095cb None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:47.027712 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeadeb5-5b74-4af2-9e36-254da6e095cb None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:47.030444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeadeb5-5b74-4af2-9e36-254da6e095cb None admin] RBAC: Adding query filter params (name=member) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:47.030783 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeadeb5-5b74-4af2-9e36-254da6e095cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:47.032400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeadeb5-5b74-4af2-9e36-254da6e095cb None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:47.043756 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 60/125] 10.222.0.41 () {56 vars in 1101 bytes} [Fri Jun 3 16:01:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:47.044592 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeadeb5-5b74-4af2-9e36-254da6e095cb None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:47.071859 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a997de-ec18-45e3-b9e3-89ec36d4ea63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:47.072632 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a997de-ec18-45e3-b9e3-89ec36d4ea63 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:47.072928 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a997de-ec18-45e3-b9e3-89ec36d4ea63 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:47.073668 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a997de-ec18-45e3-b9e3-89ec36d4ea63 None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:47.074445 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a997de-ec18-45e3-b9e3-89ec36d4ea63 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:47.074762 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a997de-ec18-45e3-b9e3-89ec36d4ea63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:47.076240 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a997de-ec18-45e3-b9e3-89ec36d4ea63 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:47.086747 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 66/126] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:01:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:47.087564 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a997de-ec18-45e3-b9e3-89ec36d4ea63 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:47.167558 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeadeb5-5b74-4af2-9e36-254da6e095cb None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:47.195267 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a997de-ec18-45e3-b9e3-89ec36d4ea63 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:47.204628 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb33e4-3e5e-4e6e-9fe1-2ad99b90c858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:47.205628 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb33e4-3e5e-4e6e-9fe1-2ad99b90c858 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:47.205922 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb33e4-3e5e-4e6e-9fe1-2ad99b90c858 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:47.206213 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb33e4-3e5e-4e6e-9fe1-2ad99b90c858 None admin] PATH_INFO: `/v3/projects/1d2895687ae2471682919de9b8fa4afb/users/5aca81affd5542cf9480c6e76c8c5478/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:47.208495 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfb33e4-3e5e-4e6e-9fe1-2ad99b90c858 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d2895687ae2471682919de9b8fa4afb, user_id=5aca81affd5542cf9480c6e76c8c5478, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:47.210085 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfb33e4-3e5e-4e6e-9fe1-2ad99b90c858 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:47.219358 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:47.231288 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 61/127] 10.222.0.41 () {58 vars in 1439 bytes} [Fri Jun 3 16:01:47 2022] PUT /identity/v3/projects/1d2895687ae2471682919de9b8fa4afb/users/5aca81affd5542cf9480c6e76c8c5478/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:01:47.236136 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb33e4-3e5e-4e6e-9fe1-2ad99b90c858 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:47.248853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434ab37-49e5-478b-ac65-6a6fe2ca30dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:47.249657 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434ab37-49e5-478b-ac65-6a6fe2ca30dd None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:47.249951 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434ab37-49e5-478b-ac65-6a6fe2ca30dd None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:47.250234 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434ab37-49e5-478b-ac65-6a6fe2ca30dd None admin] PATH_INFO: `/v3/users/03685faa84b0451694258e03a8a8dc1e` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:47.267344 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:47.272092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f434ab37-49e5-478b-ac65-6a6fe2ca30dd None admin] RBAC: Authorizing `identity:get_user(user_id=03685faa84b0451694258e03a8a8dc1e)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:47.273617 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f434ab37-49e5-478b-ac65-6a6fe2ca30dd None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:47.281362 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 67/128] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:01:47 2022] GET /identity/v3/users/03685faa84b0451694258e03a8a8dc1e => generated 289 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:47.288938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434ab37-49e5-478b-ac65-6a6fe2ca30dd None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:47.453650 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb33e4-3e5e-4e6e-9fe1-2ad99b90c858 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:47.474425 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434ab37-49e5-478b-ac65-6a6fe2ca30dd None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:47.511554 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f2df35-5047-4419-a618-536523217b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:47.513983 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2df35-5047-4419-a618-536523217b41 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:47.513983 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2df35-5047-4419-a618-536523217b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:47.513983 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2df35-5047-4419-a618-536523217b41 None admin] PATH_INFO: `/v3/users/f6a0ad4d1a084c0db8eb52bd4ed3a69a` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:47.514226 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f2df35-5047-4419-a618-536523217b41 None admin] RBAC: Authorizing `identity:get_user(user_id=f6a0ad4d1a084c0db8eb52bd4ed3a69a)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:47.516242 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f2df35-5047-4419-a618-536523217b41 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:47.518008 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 68/129] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:01:47 2022] GET /identity/v3/users/f6a0ad4d1a084c0db8eb52bd4ed3a69a => generated 275 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:47.518008 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f2df35-5047-4419-a618-536523217b41 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:47.525457 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773c417-84f8-4ff7-88cf-c26c4d68f4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:47.526372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773c417-84f8-4ff7-88cf-c26c4d68f4c6 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:47.526649 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773c417-84f8-4ff7-88cf-c26c4d68f4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:47.526926 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773c417-84f8-4ff7-88cf-c26c4d68f4c6 None admin] PATH_INFO: `/v3/users/f6a0ad4d1a084c0db8eb52bd4ed3a69a` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:47.528208 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773c417-84f8-4ff7-88cf-c26c4d68f4c6 None admin] RBAC: Authorizing `identity:get_user(user_id=f6a0ad4d1a084c0db8eb52bd4ed3a69a)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:47.529922 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773c417-84f8-4ff7-88cf-c26c4d68f4c6 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:47.531914 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 62/130] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:01:47 2022] GET /identity/v3/users/f6a0ad4d1a084c0db8eb52bd4ed3a69a => generated 275 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:47.532703 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773c417-84f8-4ff7-88cf-c26c4d68f4c6 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:47.570742 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:47.623548 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f2df35-5047-4419-a618-536523217b41 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:47.659199 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:47.660309 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce680ee-d7df-4d19-8628-a48c3a3695dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:47.660977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce680ee-d7df-4d19-8628-a48c3a3695dc None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:47.661266 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce680ee-d7df-4d19-8628-a48c3a3695dc None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:47.661547 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce680ee-d7df-4d19-8628-a48c3a3695dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:47.691000 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773c417-84f8-4ff7-88cf-c26c4d68f4c6 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:47.725523 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bafd24-f770-45a7-bf24-a3a35c4a3a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:47.726309 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bafd24-f770-45a7-bf24-a3a35c4a3a53 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:47.726593 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bafd24-f770-45a7-bf24-a3a35c4a3a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:47.726871 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bafd24-f770-45a7-bf24-a3a35c4a3a53 None admin] PATH_INFO: `/v3/projects/1d2895687ae2471682919de9b8fa4afb` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:47.728108 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bafd24-f770-45a7-bf24-a3a35c4a3a53 None admin] RBAC: Authorizing `identity:get_project(project_id=1d2895687ae2471682919de9b8fa4afb)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:47.729570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bafd24-f770-45a7-bf24-a3a35c4a3a53 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:47.731585 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 63/131] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:01:47 2022] GET /identity/v3/projects/1d2895687ae2471682919de9b8fa4afb => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:47.732363 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bafd24-f770-45a7-bf24-a3a35c4a3a53 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:47.842799 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bafd24-f770-45a7-bf24-a3a35c4a3a53 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:47.856153 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce680ee-d7df-4d19-8628-a48c3a3695dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:47.857647 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce680ee-d7df-4d19-8628-a48c3a3695dc None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:47.879932 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c179f-f942-46c2-a9f8-ddd20647568b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:47.881623 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c179f-f942-46c2-a9f8-ddd20647568b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:47.881913 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c179f-f942-46c2-a9f8-ddd20647568b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:47.882196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c179f-f942-46c2-a9f8-ddd20647568b None admin] PATH_INFO: `/v3/projects/0be3fd66a8a74fb8861ddd3788b1f6ca` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:47.883434 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c179f-f942-46c2-a9f8-ddd20647568b None admin] RBAC: Authorizing `identity:get_project(project_id=0be3fd66a8a74fb8861ddd3788b1f6ca)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:47.885106 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c179f-f942-46c2-a9f8-ddd20647568b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:47.886955 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 64/132] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:01:47 2022] GET /identity/v3/projects/0be3fd66a8a74fb8861ddd3788b1f6ca => generated 312 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:47.887729 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c179f-f942-46c2-a9f8-ddd20647568b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:47.913574 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:48.003609 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c179f-f942-46c2-a9f8-ddd20647568b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:48.012627 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:48.044175 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef07cb5-f760-4858-90d9-0e4296a526ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:48.045263 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef07cb5-f760-4858-90d9-0e4296a526ef None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:48.045555 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef07cb5-f760-4858-90d9-0e4296a526ef None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:48.045843 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef07cb5-f760-4858-90d9-0e4296a526ef None admin] PATH_INFO: `/v3/projects/1d2895687ae2471682919de9b8fa4afb/users/f6a0ad4d1a084c0db8eb52bd4ed3a69a/roles/2248ac4d70c34e459b8942dec0d7656b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:48.048296 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef07cb5-f760-4858-90d9-0e4296a526ef None admin] RBAC: Authorizing `identity:create_grant(project_id=1d2895687ae2471682919de9b8fa4afb, user_id=f6a0ad4d1a084c0db8eb52bd4ed3a69a, role_id=2248ac4d70c34e459b8942dec0d7656b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:48.049781 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef07cb5-f760-4858-90d9-0e4296a526ef None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:48.067069 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 65/133] 10.222.0.41 () {58 vars in 1439 bytes} [Fri Jun 3 16:01:47 2022] PUT /identity/v3/projects/1d2895687ae2471682919de9b8fa4afb/users/f6a0ad4d1a084c0db8eb52bd4ed3a69a/roles/2248ac4d70c34e459b8942dec0d7656b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:01:48.067954 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef07cb5-f760-4858-90d9-0e4296a526ef None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:48.179203 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 69/134] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:01:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 661 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 16:01:48.185175 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce680ee-d7df-4d19-8628-a48c3a3695dc None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:48.231805 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:48.295525 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef07cb5-f760-4858-90d9-0e4296a526ef None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:48.394959 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d5295-fcfa-4d6d-b62e-d2485cdf8fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:48.396335 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d5295-fcfa-4d6d-b62e-d2485cdf8fdd None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:48.397702 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d5295-fcfa-4d6d-b62e-d2485cdf8fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:48.398367 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d5295-fcfa-4d6d-b62e-d2485cdf8fdd None admin] PATH_INFO: `/v3/projects/0be3fd66a8a74fb8861ddd3788b1f6ca/users/f6a0ad4d1a084c0db8eb52bd4ed3a69a/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:48.400909 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1d5295-fcfa-4d6d-b62e-d2485cdf8fdd None admin] RBAC: Authorizing `identity:create_grant(project_id=0be3fd66a8a74fb8861ddd3788b1f6ca, user_id=f6a0ad4d1a084c0db8eb52bd4ed3a69a, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:48.402897 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1d5295-fcfa-4d6d-b62e-d2485cdf8fdd None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:48.445699 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 66/135] 10.222.0.41 () {58 vars in 1439 bytes} [Fri Jun 3 16:01:48 2022] PUT /identity/v3/projects/0be3fd66a8a74fb8861ddd3788b1f6ca/users/f6a0ad4d1a084c0db8eb52bd4ed3a69a/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:01:48.452859 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce680ee-d7df-4d19-8628-a48c3a3695dc None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:48.501678 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a227b2-ef7e-4811-940a-d81185173b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:48.502566 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a227b2-ef7e-4811-940a-d81185173b5e None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:48.502853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a227b2-ef7e-4811-940a-d81185173b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:48.503186 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a227b2-ef7e-4811-940a-d81185173b5e None admin] PATH_INFO: `/v3/projects/1d2895687ae2471682919de9b8fa4afb` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:48.504458 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a227b2-ef7e-4811-940a-d81185173b5e None admin] RBAC: Authorizing `identity:get_project(project_id=1d2895687ae2471682919de9b8fa4afb)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:48.506488 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a227b2-ef7e-4811-940a-d81185173b5e None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:48.508464 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 70/136] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:01:48 2022] GET /identity/v3/projects/1d2895687ae2471682919de9b8fa4afb => generated 298 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:01:48.514312 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d5295-fcfa-4d6d-b62e-d2485cdf8fdd None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:48.659953 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d5295-fcfa-4d6d-b62e-d2485cdf8fdd None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:48.702521 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54a61cd-9210-4fdc-912e-e2ebe806a949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:48.703183 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a61cd-9210-4fdc-912e-e2ebe806a949 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:48.703506 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a61cd-9210-4fdc-912e-e2ebe806a949 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:48.703783 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a61cd-9210-4fdc-912e-e2ebe806a949 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:48.704632 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54a61cd-9210-4fdc-912e-e2ebe806a949 None admin] RBAC: Adding query filter params (role.id=23e883fa91ba4c4f8fa46a1414e9761a, scope.project.id=1d2895687ae2471682919de9b8fa4afb, user.id=03685faa84b0451694258e03a8a8dc1e) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:48.704947 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54a61cd-9210-4fdc-912e-e2ebe806a949 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:48.706427 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54a61cd-9210-4fdc-912e-e2ebe806a949 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:48.708524 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:48.715649 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 67/137] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:01:48 2022] GET /identity/v3/role_assignments?user.id=03685faa84b0451694258e03a8a8dc1e&scope.project.id=1d2895687ae2471682919de9b8fa4afb&role.id=23e883fa91ba4c4f8fa46a1414e9761a => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:49.426891 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6635b4ba-c2c7-473c-85af-db58bfc6f485 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:49.427270 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6635b4ba-c2c7-473c-85af-db58bfc6f485 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:49.427547 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6635b4ba-c2c7-473c-85af-db58bfc6f485 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:49.429680 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 71/138] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:01:49.440729 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a49ed-f828-483b-9418-86b8f7f6d1a2 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:49.441212 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a49ed-f828-483b-9418-86b8f7f6d1a2 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:49.441288 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a49ed-f828-483b-9418-86b8f7f6d1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:49.462618 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-089a49ed-f828-483b-9418-86b8f7f6d1a2 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:49.463837 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-089a49ed-f828-483b-9418-86b8f7f6d1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:49.544128 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 68/139] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:01:49.584629 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6635b4ba-c2c7-473c-85af-db58bfc6f485 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:49.607783 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:49.712198 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6635b4ba-c2c7-473c-85af-db58bfc6f485 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:49.753428 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dffe342-b4dd-4a0e-8abd-f093bf8343c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:49.754230 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dffe342-b4dd-4a0e-8abd-f093bf8343c6 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:49.754515 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dffe342-b4dd-4a0e-8abd-f093bf8343c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:49.754792 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dffe342-b4dd-4a0e-8abd-f093bf8343c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:49.760002 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dffe342-b4dd-4a0e-8abd-f093bf8343c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:49.761462 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dffe342-b4dd-4a0e-8abd-f093bf8343c6 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:49.763084 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 72/140] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:01:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:49.768573 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a49ed-f828-483b-9418-86b8f7f6d1a2 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:49.832092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6cafd3-bc62-40aa-975c-822dba68ea5e None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:49.832276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6cafd3-bc62-40aa-975c-822dba68ea5e None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:49.832562 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6cafd3-bc62-40aa-975c-822dba68ea5e None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:49.834455 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 73/141] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 16:01:49.840145 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88aeee-807b-460e-b218-f7f164a289ab None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:49.840145 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88aeee-807b-460e-b218-f7f164a289ab None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:49.840145 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88aeee-807b-460e-b218-f7f164a289ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:49.868526 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-df88aeee-807b-460e-b218-f7f164a289ab None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:49.869723 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-df88aeee-807b-460e-b218-f7f164a289ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:49.911528 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a49ed-f828-483b-9418-86b8f7f6d1a2 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:49.954327 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:49.971698 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f74a9-822d-4249-82b9-3d0fbdd98dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:49.972387 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f74a9-822d-4249-82b9-3d0fbdd98dd0 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:49.972680 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f74a9-822d-4249-82b9-3d0fbdd98dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:49.972958 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f74a9-822d-4249-82b9-3d0fbdd98dd0 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:49.973694 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8f74a9-822d-4249-82b9-3d0fbdd98dd0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:49.975415 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8f74a9-822d-4249-82b9-3d0fbdd98dd0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:50.001329 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 74/142] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 16:01:50.021961 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 69/143] 10.222.0.41 () {60 vars in 1132 bytes} [Fri Jun 3 16:01:49 2022] POST /identity/v3/users => generated 283 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:01:50.045708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df88aeee-807b-460e-b218-f7f164a289ab None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:50.067616 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:50.191094 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df88aeee-807b-460e-b218-f7f164a289ab None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:50.230398 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bced72-980e-473b-99b7-0651f48fa9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:50.231201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bced72-980e-473b-99b7-0651f48fa9dd None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:50.231536 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bced72-980e-473b-99b7-0651f48fa9dd None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:50.231812 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bced72-980e-473b-99b7-0651f48fa9dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:50.244374 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bced72-980e-473b-99b7-0651f48fa9dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:01:50.253644 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-19bced72-980e-473b-99b7-0651f48fa9dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 16:01:50.253644 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:01:50.253644 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:01:50.253644 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:01:50.253644 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:01:50.253644 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:01:50.253644 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:01:50.253644 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:01:50.253644 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:01:50.253644 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:01:50.253644 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:01:50.253644 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:01:50.253644 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:01:50.253644 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:01:50.253644 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:01:50.253644 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:01:50.254499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:01:50.254499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:01:50.254499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:01:50.254499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:01:50.254499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:01:50.254499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:01:50.254499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:01:50.254499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:01:50.254499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:01:50.254499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:01:50.254499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:01:50.254499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:01:50.254499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:01:50.254499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:01:50.254499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:01:50.254499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:01:50.255245 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:01:50.255245 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:01:50.255245 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:01:50.255245 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:01:50.255245 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:01:50.255245 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:01:50.255245 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:01:50.255245 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:01:50.255245 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:01:50.255245 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:01:50.255245 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 16:01:50.255245 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:01:50.255245 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 75/144] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:01:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:01:50.262363 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f74a9-822d-4249-82b9-3d0fbdd98dd0 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:50.404772 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f74a9-822d-4249-82b9-3d0fbdd98dd0 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:50.449637 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c23cc8-f516-4d6a-801d-830d4c9b400f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:50.450219 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c23cc8-f516-4d6a-801d-830d4c9b400f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:50.450504 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c23cc8-f516-4d6a-801d-830d4c9b400f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:50.450784 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c23cc8-f516-4d6a-801d-830d4c9b400f None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:50.451570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c23cc8-f516-4d6a-801d-830d4c9b400f None admin] RBAC: Adding query filter params (name=member) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:50.451890 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c23cc8-f516-4d6a-801d-830d4c9b400f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:50.453274 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c23cc8-f516-4d6a-801d-830d4c9b400f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:50.467106 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 70/145] 10.222.0.41 () {56 vars in 1101 bytes} [Fri Jun 3 16:01:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:50.472579 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bced72-980e-473b-99b7-0651f48fa9dd None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:50.588138 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:50.615918 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bced72-980e-473b-99b7-0651f48fa9dd None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:50.626205 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9485e4-0ff2-4cd3-bd7d-c940e471c994 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:50.626493 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9485e4-0ff2-4cd3-bd7d-c940e471c994 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:50.626775 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9485e4-0ff2-4cd3-bd7d-c940e471c994 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:50.628432 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 71/146] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:01:50.633923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e90ed-e5c1-42e2-98fe-3478a7d997a3 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:50.634211 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e90ed-e5c1-42e2-98fe-3478a7d997a3 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:50.634489 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e90ed-e5c1-42e2-98fe-3478a7d997a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:50.651624 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68893f09-54c6-4f6d-bb94-2054549d9647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:50.652507 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68893f09-54c6-4f6d-bb94-2054549d9647 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:50.652789 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68893f09-54c6-4f6d-bb94-2054549d9647 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:50.653066 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68893f09-54c6-4f6d-bb94-2054549d9647 None admin] PATH_INFO: `/v3/users/f6a0ad4d1a084c0db8eb52bd4ed3a69a` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:50.654536 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68893f09-54c6-4f6d-bb94-2054549d9647 None admin] RBAC: Authorizing `identity:get_user(user_id=f6a0ad4d1a084c0db8eb52bd4ed3a69a)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:50.656079 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68893f09-54c6-4f6d-bb94-2054549d9647 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:50.657757 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 76/147] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:01:50 2022] GET /identity/v3/users/f6a0ad4d1a084c0db8eb52bd4ed3a69a => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:50.663109 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68893f09-54c6-4f6d-bb94-2054549d9647 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:50.665685 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c40e90ed-e5c1-42e2-98fe-3478a7d997a3 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:50.666866 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c40e90ed-e5c1-42e2-98fe-3478a7d997a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:50.730531 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:50.767264 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68893f09-54c6-4f6d-bb94-2054549d9647 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:50.798516 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 72/148] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:01:50.801797 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5648c2a-425c-461a-833c-ed2e1a1a1a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:50.803125 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5648c2a-425c-461a-833c-ed2e1a1a1a54 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:50.803573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5648c2a-425c-461a-833c-ed2e1a1a1a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:50.803987 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5648c2a-425c-461a-833c-ed2e1a1a1a54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:50.819681 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:50.845386 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e90ed-e5c1-42e2-98fe-3478a7d997a3 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:50.876635 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5648c2a-425c-461a-833c-ed2e1a1a1a54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:50.878124 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5648c2a-425c-461a-833c-ed2e1a1a1a54 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:50.981375 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 77/149] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:01:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 03 16:01:50.986667 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5648c2a-425c-461a-833c-ed2e1a1a1a54 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:51.034657 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:51.089595 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e90ed-e5c1-42e2-98fe-3478a7d997a3 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:51.098359 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5648c2a-425c-461a-833c-ed2e1a1a1a54 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:51.133558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e885eb-ff62-4d89-a0ac-ef59757117b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:51.134429 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e885eb-ff62-4d89-a0ac-ef59757117b0 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:51.134710 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e885eb-ff62-4d89-a0ac-ef59757117b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:51.134987 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e885eb-ff62-4d89-a0ac-ef59757117b0 None admin] PATH_INFO: `/v3/projects/1d2895687ae2471682919de9b8fa4afb` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:51.136215 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e885eb-ff62-4d89-a0ac-ef59757117b0 None admin] RBAC: Authorizing `identity:get_project(project_id=1d2895687ae2471682919de9b8fa4afb)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:51.137642 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e885eb-ff62-4d89-a0ac-ef59757117b0 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:51.139531 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 78/150] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:01:50 2022] GET /identity/v3/projects/1d2895687ae2471682919de9b8fa4afb => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:01:51.144996 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e885eb-ff62-4d89-a0ac-ef59757117b0 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:51.174148 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bfe576-2394-4a14-a044-00f17b9a007d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:51.174962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfe576-2394-4a14-a044-00f17b9a007d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:51.175243 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfe576-2394-4a14-a044-00f17b9a007d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:51.175546 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfe576-2394-4a14-a044-00f17b9a007d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:51.202505 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bfe576-2394-4a14-a044-00f17b9a007d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:01:51.210923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-33bfe576-2394-4a14-a044-00f17b9a007d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:01:51.210923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:01:51.210923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:01:51.210923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:01:51.210923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:01:51.210923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:01:51.210923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:01:51.210923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:01:51.210923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:01:51.210923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:01:51.210923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:01:51.210923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:01:51.210923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:01:51.210923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:01:51.210923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:01:51.210923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:01:51.211892 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:01:51.211892 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:01:51.211892 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:01:51.211892 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:01:51.211892 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:01:51.211892 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:01:51.211892 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:01:51.211892 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:01:51.211892 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:01:51.211892 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:01:51.211892 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:01:51.211892 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:01:51.211892 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:01:51.211892 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:01:51.211892 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:01:51.211892 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:01:51.212645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:01:51.212645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:01:51.212645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:01:51.212645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:01:51.212645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:01:51.212645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:01:51.212645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:01:51.212645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:01:51.212645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:01:51.212645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:01:51.212645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:01:51.212645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:01:51.216653 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 73/151] 10.222.0.41 () {56 vars in 1096 bytes} [Fri Jun 3 16:01:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:01:51.226596 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bfe576-2394-4a14-a044-00f17b9a007d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:51.298982 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:51.321831 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e885eb-ff62-4d89-a0ac-ef59757117b0 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:51.355792 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:51.408537 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bfe576-2394-4a14-a044-00f17b9a007d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:51.413081 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31001e79-cda4-4823-9c8e-a6060c88c8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:51.413764 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31001e79-cda4-4823-9c8e-a6060c88c8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:51.414053 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31001e79-cda4-4823-9c8e-a6060c88c8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:51.419366 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31001e79-cda4-4823-9c8e-a6060c88c8b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:51.419366 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31001e79-cda4-4823-9c8e-a6060c88c8b4 None admin] RBAC: Adding query filter params (role.id=a8e4174e3572497f9284c709a119dcff, scope.project.id=1d2895687ae2471682919de9b8fa4afb, user.id=f6a0ad4d1a084c0db8eb52bd4ed3a69a) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:51.423619 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31001e79-cda4-4823-9c8e-a6060c88c8b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:51.425164 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31001e79-cda4-4823-9c8e-a6060c88c8b4 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:51.436223 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 79/152] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:01:51 2022] GET /identity/v3/role_assignments?user.id=f6a0ad4d1a084c0db8eb52bd4ed3a69a&scope.project.id=1d2895687ae2471682919de9b8fa4afb&role.id=a8e4174e3572497f9284c709a119dcff => generated 608 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:01:51.455823 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47beb209-8d4c-4917-8b6e-a4a89fd16463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:51.459315 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47beb209-8d4c-4917-8b6e-a4a89fd16463 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:51.459315 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47beb209-8d4c-4917-8b6e-a4a89fd16463 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:51.459315 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47beb209-8d4c-4917-8b6e-a4a89fd16463 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:51.459315 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47beb209-8d4c-4917-8b6e-a4a89fd16463 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:51.459315 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47beb209-8d4c-4917-8b6e-a4a89fd16463 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:51.460139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47beb209-8d4c-4917-8b6e-a4a89fd16463 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:51.471627 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 74/153] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:01:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:01:51.487539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31001e79-cda4-4823-9c8e-a6060c88c8b4 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:51.654983 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31001e79-cda4-4823-9c8e-a6060c88c8b4 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:51.670665 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364d5a1-51ba-42c9-b2d1-5c7d1545b154 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:51.670972 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364d5a1-51ba-42c9-b2d1-5c7d1545b154 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:51.671231 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364d5a1-51ba-42c9-b2d1-5c7d1545b154 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:51.672621 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 75/154] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 16:01:51.678504 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e432d4-0e5d-4fba-a14b-8764fb2a1397 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:51.678815 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e432d4-0e5d-4fba-a14b-8764fb2a1397 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:51.679061 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e432d4-0e5d-4fba-a14b-8764fb2a1397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:51.710112 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c0e432d4-0e5d-4fba-a14b-8764fb2a1397 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:51.713587 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c0e432d4-0e5d-4fba-a14b-8764fb2a1397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:51.736876 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3fe319-8a27-4f19-86ad-dbb29ad6d62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:51.737792 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fe319-8a27-4f19-86ad-dbb29ad6d62a None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:51.738078 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fe319-8a27-4f19-86ad-dbb29ad6d62a None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:51.738355 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fe319-8a27-4f19-86ad-dbb29ad6d62a None admin] PATH_INFO: `/v3/users/5aca81affd5542cf9480c6e76c8c5478` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:51.741853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3fe319-8a27-4f19-86ad-dbb29ad6d62a None admin] RBAC: Authorizing `identity:get_user(user_id=5aca81affd5542cf9480c6e76c8c5478)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:51.748476 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3fe319-8a27-4f19-86ad-dbb29ad6d62a None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:51.753924 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 80/155] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:01:51 2022] GET /identity/v3/users/5aca81affd5542cf9480c6e76c8c5478 => generated 247 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:51.763573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3fe319-8a27-4f19-86ad-dbb29ad6d62a None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:51.844960 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 76/156] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 16:01:51.859671 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:51.910411 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e432d4-0e5d-4fba-a14b-8764fb2a1397 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:51.963149 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3fe319-8a27-4f19-86ad-dbb29ad6d62a None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:51.973039 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:52.018716 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b511ba4-c062-4ccd-8e49-e369ffc9f44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:52.019409 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b511ba4-c062-4ccd-8e49-e369ffc9f44f None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:52.019701 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b511ba4-c062-4ccd-8e49-e369ffc9f44f None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:52.020021 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b511ba4-c062-4ccd-8e49-e369ffc9f44f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:52.111623 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e432d4-0e5d-4fba-a14b-8764fb2a1397 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:52.132606 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b511ba4-c062-4ccd-8e49-e369ffc9f44f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:52.134072 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b511ba4-c062-4ccd-8e49-e369ffc9f44f None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:52.175080 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e0e75-af11-48ef-9dc8-8dcf1869a0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:52.176346 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e0e75-af11-48ef-9dc8-8dcf1869a0a7 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:52.176850 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e0e75-af11-48ef-9dc8-8dcf1869a0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:52.178103 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:52.178376 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e0e75-af11-48ef-9dc8-8dcf1869a0a7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:52.207736 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77e0e75-af11-48ef-9dc8-8dcf1869a0a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:01:52.216897 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-c77e0e75-af11-48ef-9dc8-8dcf1869a0a7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 16:01:52.216897 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:01:52.216897 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:01:52.216897 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:01:52.216897 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:01:52.216897 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:01:52.216897 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:01:52.216897 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:01:52.216897 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:01:52.216897 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:01:52.216897 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:01:52.216897 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:01:52.216897 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:01:52.216897 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:01:52.216897 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:01:52.216897 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:01:52.217902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:01:52.217902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:01:52.217902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:01:52.217902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:01:52.217902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:01:52.217902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:01:52.217902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:01:52.217902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:01:52.217902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:01:52.217902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:01:52.217902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:01:52.217902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:01:52.217902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:01:52.217902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:01:52.217902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:01:52.217902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:01:52.218651 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:01:52.218651 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:01:52.218651 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:01:52.218651 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:01:52.218651 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:01:52.218651 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:01:52.218651 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:01:52.218651 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:01:52.218651 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:01:52.218651 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:01:52.218651 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 16:01:52.218651 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:01:52.230127 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 77/157] 10.222.0.41 () {56 vars in 1114 bytes} [Fri Jun 3 16:01:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:01:52.235837 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e0e75-af11-48ef-9dc8-8dcf1869a0a7 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:52.263179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 81/158] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:01:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 16:01:52.268684 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b511ba4-c062-4ccd-8e49-e369ffc9f44f None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:52.385365 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b511ba4-c062-4ccd-8e49-e369ffc9f44f None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:52.402289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:52.429256 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e0e75-af11-48ef-9dc8-8dcf1869a0a7 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:52.436962 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13679971-d09c-479b-9d1b-153dc16e2964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:52.437854 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13679971-d09c-479b-9d1b-153dc16e2964 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:52.438139 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13679971-d09c-479b-9d1b-153dc16e2964 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:52.438438 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13679971-d09c-479b-9d1b-153dc16e2964 None admin] PATH_INFO: `/v3/projects/1d2895687ae2471682919de9b8fa4afb` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:52.439679 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13679971-d09c-479b-9d1b-153dc16e2964 None admin] RBAC: Authorizing `identity:get_project(project_id=1d2895687ae2471682919de9b8fa4afb)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:52.441132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13679971-d09c-479b-9d1b-153dc16e2964 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:52.442849 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 82/159] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:01:52 2022] GET /identity/v3/projects/1d2895687ae2471682919de9b8fa4afb => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:01:52.448731 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13679971-d09c-479b-9d1b-153dc16e2964 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:52.463659 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:52.523173 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2263a98-e2f4-472f-a6fb-57944103cca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:52.523923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2263a98-e2f4-472f-a6fb-57944103cca6 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:52.524208 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2263a98-e2f4-472f-a6fb-57944103cca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:52.524479 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2263a98-e2f4-472f-a6fb-57944103cca6 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:52.525248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2263a98-e2f4-472f-a6fb-57944103cca6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:52.525554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2263a98-e2f4-472f-a6fb-57944103cca6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:52.526940 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2263a98-e2f4-472f-a6fb-57944103cca6 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:52.557706 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 78/160] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:01:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:52.559064 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a24840-0814-4085-868b-8c6724d5d6d5 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:52.559373 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a24840-0814-4085-868b-8c6724d5d6d5 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:52.559546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13679971-d09c-479b-9d1b-153dc16e2964 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:52.559657 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a24840-0814-4085-868b-8c6724d5d6d5 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:52.561299 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 79/161] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:01:52.567018 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac1b87-9843-46b6-afc6-39c6a40422d3 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:52.567330 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac1b87-9843-46b6-afc6-39c6a40422d3 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:52.567611 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac1b87-9843-46b6-afc6-39c6a40422d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:52.585025 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-22ac1b87-9843-46b6-afc6-39c6a40422d3 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:52.586245 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-22ac1b87-9843-46b6-afc6-39c6a40422d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:52.601648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37981230-710a-4def-9d2a-6b635bee90bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:52.602706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37981230-710a-4def-9d2a-6b635bee90bc None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:52.603004 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37981230-710a-4def-9d2a-6b635bee90bc None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:52.603286 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37981230-710a-4def-9d2a-6b635bee90bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:52.604409 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37981230-710a-4def-9d2a-6b635bee90bc None admin] RBAC: Adding query filter params (role.id=75936a608b8149099ff54e317f3a6740, scope.project.id=1d2895687ae2471682919de9b8fa4afb, user.id=5aca81affd5542cf9480c6e76c8c5478) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:52.604723 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37981230-710a-4def-9d2a-6b635bee90bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:52.606695 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37981230-710a-4def-9d2a-6b635bee90bc None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:52.619426 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 83/162] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:01:52 2022] GET /identity/v3/role_assignments?user.id=5aca81affd5542cf9480c6e76c8c5478&scope.project.id=1d2895687ae2471682919de9b8fa4afb&role.id=75936a608b8149099ff54e317f3a6740 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:01:52.621556 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37981230-710a-4def-9d2a-6b635bee90bc None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:52.703986 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 80/163] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:01:52.751831 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:52.752620 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac1b87-9843-46b6-afc6-39c6a40422d3 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:52.807725 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37981230-710a-4def-9d2a-6b635bee90bc None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:52.866677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af03f8b-7ea0-4add-b499-c57f3434c4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:52.867603 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af03f8b-7ea0-4add-b499-c57f3434c4ff None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:52.867885 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af03f8b-7ea0-4add-b499-c57f3434c4ff None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:52.868160 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af03f8b-7ea0-4add-b499-c57f3434c4ff None admin] PATH_INFO: `/v3/users/f6a0ad4d1a084c0db8eb52bd4ed3a69a` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:52.869493 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af03f8b-7ea0-4add-b499-c57f3434c4ff None admin] RBAC: Authorizing `identity:get_user(user_id=f6a0ad4d1a084c0db8eb52bd4ed3a69a)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:52.871093 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af03f8b-7ea0-4add-b499-c57f3434c4ff None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:52.872826 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 84/164] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:01:52 2022] GET /identity/v3/users/f6a0ad4d1a084c0db8eb52bd4ed3a69a => generated 275 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:52.878063 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af03f8b-7ea0-4add-b499-c57f3434c4ff None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:52.967739 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:53.019654 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af03f8b-7ea0-4add-b499-c57f3434c4ff None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:53.037792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac1b87-9843-46b6-afc6-39c6a40422d3 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:53.060469 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f0969-646b-44d2-ba97-25f64086befb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:53.061294 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f0969-646b-44d2-ba97-25f64086befb None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:53.061576 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f0969-646b-44d2-ba97-25f64086befb None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:53.061856 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f0969-646b-44d2-ba97-25f64086befb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:53.095709 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:53.124944 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c2b2bd-de9b-4e16-9ae4-f650a1bc8a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:53.125754 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2b2bd-de9b-4e16-9ae4-f650a1bc8a95 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:53.126036 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2b2bd-de9b-4e16-9ae4-f650a1bc8a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:53.126311 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2b2bd-de9b-4e16-9ae4-f650a1bc8a95 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:53.136154 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f0969-646b-44d2-ba97-25f64086befb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:53.137694 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f0969-646b-44d2-ba97-25f64086befb None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:53.151889 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c2b2bd-de9b-4e16-9ae4-f650a1bc8a95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:01:53.159952 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-d8c2b2bd-de9b-4e16-9ae4-f650a1bc8a95 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 16:01:53.159952 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:01:53.159952 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:01:53.159952 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:01:53.159952 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:01:53.159952 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:01:53.159952 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:01:53.159952 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:01:53.159952 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:01:53.159952 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:01:53.159952 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:01:53.159952 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:01:53.159952 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:01:53.159952 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:01:53.159952 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:01:53.159952 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:01:53.160905 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:01:53.160905 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:01:53.160905 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:01:53.160905 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:01:53.160905 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:01:53.160905 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:01:53.160905 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:01:53.160905 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:01:53.160905 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:01:53.160905 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:01:53.160905 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:01:53.160905 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:01:53.160905 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:01:53.160905 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:01:53.160905 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:01:53.160905 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:01:53.161870 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:01:53.161870 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:01:53.161870 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:01:53.161870 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:01:53.161870 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:01:53.161870 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:01:53.161870 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:01:53.161870 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:01:53.161870 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:01:53.161870 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:01:53.161870 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 16:01:53.161870 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:01:53.161870 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 81/165] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:01:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 409 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:01:53.162509 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f7a0b0-8de9-4bdc-a284-845783ec146b None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:53.162890 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f7a0b0-8de9-4bdc-a284-845783ec146b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:53.162978 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f7a0b0-8de9-4bdc-a284-845783ec146b None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:53.172642 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 82/166] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:53 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:01:53.173396 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f7a0b0-8de9-4bdc-a284-845783ec146b None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:53.308043 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f7a0b0-8de9-4bdc-a284-845783ec146b None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:53.323350 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 85/167] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:01:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 16:01:53.324655 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5181dc-a2c2-4858-aed0-7de7ba3a27a9 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:53.324946 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5181dc-a2c2-4858-aed0-7de7ba3a27a9 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:53.325225 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5181dc-a2c2-4858-aed0-7de7ba3a27a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:53.346535 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab9cb2-e7bd-4411-b521-dc468ec9335b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:53.347207 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab9cb2-e7bd-4411-b521-dc468ec9335b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:53.347530 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab9cb2-e7bd-4411-b521-dc468ec9335b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:53.347815 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab9cb2-e7bd-4411-b521-dc468ec9335b None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:53.348115 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-1f5181dc-a2c2-4858-aed0-7de7ba3a27a9 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:53.348574 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdab9cb2-e7bd-4411-b521-dc468ec9335b None admin] RBAC: Adding query filter params (name=member) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:53.348894 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdab9cb2-e7bd-4411-b521-dc468ec9335b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:53.349347 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-1f5181dc-a2c2-4858-aed0-7de7ba3a27a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:53.350359 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdab9cb2-e7bd-4411-b521-dc468ec9335b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:53.360876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 83/168] 10.222.0.41 () {56 vars in 1101 bytes} [Fri Jun 3 16:01:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:53.362222 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab9cb2-e7bd-4411-b521-dc468ec9335b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:53.375629 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:53.386129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:53.436478 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 86/169] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:01:53.437672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5181dc-a2c2-4858-aed0-7de7ba3a27a9 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:53.477704 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab9cb2-e7bd-4411-b521-dc468ec9335b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:53.516135 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff89b8e-ea45-4ef8-9cb7-effa9da51f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:53.516980 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff89b8e-ea45-4ef8-9cb7-effa9da51f6d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:53.517267 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff89b8e-ea45-4ef8-9cb7-effa9da51f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:53.517594 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff89b8e-ea45-4ef8-9cb7-effa9da51f6d None admin] PATH_INFO: `/v3/projects/1d2895687ae2471682919de9b8fa4afb` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:53.522161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff89b8e-ea45-4ef8-9cb7-effa9da51f6d None admin] RBAC: Authorizing `identity:get_project(project_id=1d2895687ae2471682919de9b8fa4afb)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:53.522161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff89b8e-ea45-4ef8-9cb7-effa9da51f6d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:53.522161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 84/170] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:01:53 2022] GET /identity/v3/projects/1d2895687ae2471682919de9b8fa4afb => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:53.522934 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff89b8e-ea45-4ef8-9cb7-effa9da51f6d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:53.595825 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:53.627764 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff89b8e-ea45-4ef8-9cb7-effa9da51f6d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:53.685590 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb2b50-08c2-4659-b499-3698e1554887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:53.687924 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb2b50-08c2-4659-b499-3698e1554887 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:53.688589 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb2b50-08c2-4659-b499-3698e1554887 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:53.689643 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb2b50-08c2-4659-b499-3698e1554887 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:53.694763 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5181dc-a2c2-4858-aed0-7de7ba3a27a9 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:53.711194 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bb2b50-08c2-4659-b499-3698e1554887 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:01:53.719361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-f7bb2b50-08c2-4659-b499-3698e1554887 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 16:01:53.719361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:01:53.719361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:01:53.719361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:01:53.719361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:01:53.719361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:01:53.719361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:01:53.719361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:01:53.719361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:01:53.719361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:01:53.719361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:01:53.719361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:01:53.719361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:01:53.719361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:01:53.719361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:01:53.719361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:01:53.720319 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:01:53.720319 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:01:53.720319 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:01:53.720319 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:01:53.720319 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:01:53.720319 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:01:53.720319 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:01:53.720319 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:01:53.720319 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:01:53.720319 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:01:53.720319 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:01:53.720319 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:01:53.720319 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:01:53.720319 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:01:53.720319 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:01:53.720319 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:01:53.721101 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:01:53.721101 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:01:53.721101 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:01:53.721101 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:01:53.721101 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:01:53.721101 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:01:53.721101 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:01:53.721101 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:01:53.721101 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:01:53.721101 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:01:53.721101 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 16:01:53.721101 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:01:53.723545 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 85/171] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:01:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:01:53.724542 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb2b50-08c2-4659-b499-3698e1554887 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:53.792170 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f227ee-56ad-4a2e-b7e2-74212b7b3225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:53.792921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f227ee-56ad-4a2e-b7e2-74212b7b3225 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:53.793210 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f227ee-56ad-4a2e-b7e2-74212b7b3225 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:53.793486 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f227ee-56ad-4a2e-b7e2-74212b7b3225 None admin] PATH_INFO: `/v3/users/f6a0ad4d1a084c0db8eb52bd4ed3a69a` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:53.794732 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f227ee-56ad-4a2e-b7e2-74212b7b3225 None admin] RBAC: Authorizing `identity:get_user(user_id=f6a0ad4d1a084c0db8eb52bd4ed3a69a)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:53.796258 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f227ee-56ad-4a2e-b7e2-74212b7b3225 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:53.798068 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 87/172] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:01:53 2022] GET /identity/v3/users/f6a0ad4d1a084c0db8eb52bd4ed3a69a => generated 275 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:53.799520 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f227ee-56ad-4a2e-b7e2-74212b7b3225 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:53.858116 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:53.892111 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb2b50-08c2-4659-b499-3698e1554887 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:53.913954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f227ee-56ad-4a2e-b7e2-74212b7b3225 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:53.949963 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5893fb35-a22b-4ff2-9335-4de404cb9742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:53.950637 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893fb35-a22b-4ff2-9335-4de404cb9742 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:53.950917 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893fb35-a22b-4ff2-9335-4de404cb9742 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:53.951497 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893fb35-a22b-4ff2-9335-4de404cb9742 None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:53.952073 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5893fb35-a22b-4ff2-9335-4de404cb9742 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:53.952386 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5893fb35-a22b-4ff2-9335-4de404cb9742 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:53.953769 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5893fb35-a22b-4ff2-9335-4de404cb9742 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:53.962546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 88/173] 10.222.0.41 () {56 vars in 1101 bytes} [Fri Jun 3 16:01:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:53.963534 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5893fb35-a22b-4ff2-9335-4de404cb9742 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:53.987708 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:54.012626 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df91e64-6eae-4629-8117-d597b5ac2718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:54.013305 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df91e64-6eae-4629-8117-d597b5ac2718 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:54.013588 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df91e64-6eae-4629-8117-d597b5ac2718 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:54.013857 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df91e64-6eae-4629-8117-d597b5ac2718 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:54.014706 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df91e64-6eae-4629-8117-d597b5ac2718 None admin] RBAC: Adding query filter params (role.id=2248ac4d70c34e459b8942dec0d7656b, scope.project.id=1d2895687ae2471682919de9b8fa4afb, user.id=f6a0ad4d1a084c0db8eb52bd4ed3a69a) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:54.015018 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df91e64-6eae-4629-8117-d597b5ac2718 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:54.016634 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df91e64-6eae-4629-8117-d597b5ac2718 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:54.038015 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 86/174] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:01:53 2022] GET /identity/v3/role_assignments?user.id=f6a0ad4d1a084c0db8eb52bd4ed3a69a&scope.project.id=1d2895687ae2471682919de9b8fa4afb&role.id=2248ac4d70c34e459b8942dec0d7656b => generated 608 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:54.039235 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df91e64-6eae-4629-8117-d597b5ac2718 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:54.069936 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5893fb35-a22b-4ff2-9335-4de404cb9742 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:54.073293 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:54.113404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea0460-d3e9-4ed5-849c-59ff6ced4c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:54.114074 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea0460-d3e9-4ed5-849c-59ff6ced4c21 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:54.114361 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea0460-d3e9-4ed5-849c-59ff6ced4c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:54.114664 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea0460-d3e9-4ed5-849c-59ff6ced4c21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:54.254583 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfea0460-d3e9-4ed5-849c-59ff6ced4c21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:54.256169 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfea0460-d3e9-4ed5-849c-59ff6ced4c21 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:54.288091 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df91e64-6eae-4629-8117-d597b5ac2718 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:54.370509 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ba6c0-f1d7-49da-b323-3f252e6386af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:54.371839 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ba6c0-f1d7-49da-b323-3f252e6386af None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:54.372319 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ba6c0-f1d7-49da-b323-3f252e6386af None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:54.372800 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ba6c0-f1d7-49da-b323-3f252e6386af None admin] PATH_INFO: `/v3/users/03685faa84b0451694258e03a8a8dc1e` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:54.380325 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050ba6c0-f1d7-49da-b323-3f252e6386af None admin] RBAC: Authorizing `identity:get_user(user_id=03685faa84b0451694258e03a8a8dc1e)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:54.382440 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050ba6c0-f1d7-49da-b323-3f252e6386af None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:54.384394 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 87/175] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:01:54 2022] GET /identity/v3/users/03685faa84b0451694258e03a8a8dc1e => generated 289 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:54.390185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ba6c0-f1d7-49da-b323-3f252e6386af None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:54.421260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 89/176] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:01:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 16:01:54.428558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea0460-d3e9-4ed5-849c-59ff6ced4c21 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:54.430891 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:54.475933 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:54.551700 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ba6c0-f1d7-49da-b323-3f252e6386af None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:54.577974 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea0460-d3e9-4ed5-849c-59ff6ced4c21 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:54.621458 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77992fdb-4249-4250-89e9-465aa2197ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:54.622258 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77992fdb-4249-4250-89e9-465aa2197ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:54.622548 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77992fdb-4249-4250-89e9-465aa2197ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:54.622836 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77992fdb-4249-4250-89e9-465aa2197ba5 None admin] PATH_INFO: `/v3/projects/0be3fd66a8a74fb8861ddd3788b1f6ca` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:54.624184 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77992fdb-4249-4250-89e9-465aa2197ba5 None admin] RBAC: Authorizing `identity:get_project(project_id=0be3fd66a8a74fb8861ddd3788b1f6ca)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:54.625620 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77992fdb-4249-4250-89e9-465aa2197ba5 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:54.627343 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 90/177] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:01:54 2022] GET /identity/v3/projects/0be3fd66a8a74fb8861ddd3788b1f6ca => generated 312 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:54.632593 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982bf09b-2b9b-4316-9561-01d9c536ef99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:54.633481 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982bf09b-2b9b-4316-9561-01d9c536ef99 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:54.633601 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77992fdb-4249-4250-89e9-465aa2197ba5 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:54.633759 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982bf09b-2b9b-4316-9561-01d9c536ef99 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:54.634150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982bf09b-2b9b-4316-9561-01d9c536ef99 None admin] PATH_INFO: `/v3/projects/1d2895687ae2471682919de9b8fa4afb` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:54.635373 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982bf09b-2b9b-4316-9561-01d9c536ef99 None admin] RBAC: Authorizing `identity:get_project(project_id=1d2895687ae2471682919de9b8fa4afb)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:54.636838 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982bf09b-2b9b-4316-9561-01d9c536ef99 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:54.638808 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 88/178] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:01:54 2022] GET /identity/v3/projects/1d2895687ae2471682919de9b8fa4afb => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:01:54.654501 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982bf09b-2b9b-4316-9561-01d9c536ef99 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:54.728706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:54.767599 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77992fdb-4249-4250-89e9-465aa2197ba5 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:54.804793 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982bf09b-2b9b-4316-9561-01d9c536ef99 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:54.812956 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda1c1e3-46b9-4b2a-8e45-d6e6359646e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:54.813622 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1c1e3-46b9-4b2a-8e45-d6e6359646e9 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:54.813909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1c1e3-46b9-4b2a-8e45-d6e6359646e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:54.814189 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1c1e3-46b9-4b2a-8e45-d6e6359646e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:54.815537 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda1c1e3-46b9-4b2a-8e45-d6e6359646e9 None admin] RBAC: Adding query filter params (role.id=a8e4174e3572497f9284c709a119dcff, scope.project.id=0be3fd66a8a74fb8861ddd3788b1f6ca, user.id=f6a0ad4d1a084c0db8eb52bd4ed3a69a) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:54.815859 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda1c1e3-46b9-4b2a-8e45-d6e6359646e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:54.818588 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda1c1e3-46b9-4b2a-8e45-d6e6359646e9 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:54.826307 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 91/179] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:01:54 2022] GET /identity/v3/role_assignments?user.id=f6a0ad4d1a084c0db8eb52bd4ed3a69a&scope.project.id=0be3fd66a8a74fb8861ddd3788b1f6ca&role.id=a8e4174e3572497f9284c709a119dcff => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:54.832766 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d59903-18dd-4d7a-b9fe-546202c1ffcb None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:54.833119 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d59903-18dd-4d7a-b9fe-546202c1ffcb None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:54.833466 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d59903-18dd-4d7a-b9fe-546202c1ffcb None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:54.835600 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 92/180] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:54 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:01:54.842260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23657724-070c-4699-bafe-dfa6ed80b4d8 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:54.842566 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23657724-070c-4699-bafe-dfa6ed80b4d8 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:54.842805 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23657724-070c-4699-bafe-dfa6ed80b4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:54.850600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:54.863078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af70c82b-adad-4997-a6e4-4be88fd1d03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:54.864420 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70c82b-adad-4997-a6e4-4be88fd1d03a None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:54.865008 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70c82b-adad-4997-a6e4-4be88fd1d03a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:54.865619 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70c82b-adad-4997-a6e4-4be88fd1d03a None admin] PATH_INFO: `/v3/projects/1d2895687ae2471682919de9b8fa4afb/users/03685faa84b0451694258e03a8a8dc1e/roles/23e883fa91ba4c4f8fa46a1414e9761a` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:54.866382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-23657724-070c-4699-bafe-dfa6ed80b4d8 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:54.867854 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-23657724-070c-4699-bafe-dfa6ed80b4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:54.868614 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af70c82b-adad-4997-a6e4-4be88fd1d03a None admin] RBAC: Authorizing `identity:create_grant(project_id=1d2895687ae2471682919de9b8fa4afb, user_id=03685faa84b0451694258e03a8a8dc1e, role_id=23e883fa91ba4c4f8fa46a1414e9761a)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:54.870512 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af70c82b-adad-4997-a6e4-4be88fd1d03a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:54.889130 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 89/181] 10.222.0.41 () {58 vars in 1439 bytes} [Fri Jun 3 16:01:54 2022] PUT /identity/v3/projects/1d2895687ae2471682919de9b8fa4afb/users/03685faa84b0451694258e03a8a8dc1e/roles/23e883fa91ba4c4f8fa46a1414e9761a => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:01:54.890716 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f0424-40ca-4464-a247-d72561c04aba None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:54.891009 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f0424-40ca-4464-a247-d72561c04aba None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:54.891287 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f0424-40ca-4464-a247-d72561c04aba None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:54.892733 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 90/182] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:01:54.899110 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38164788-befa-4d78-9d5f-3bc1309352de None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:54.899484 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38164788-befa-4d78-9d5f-3bc1309352de None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:54.899771 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38164788-befa-4d78-9d5f-3bc1309352de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:54.917995 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-38164788-befa-4d78-9d5f-3bc1309352de None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:54.919235 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-38164788-befa-4d78-9d5f-3bc1309352de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:54.960773 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 93/183] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:01:54.962712 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d624b7-b544-417d-9cd4-38d17452b08a None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:54.963190 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d624b7-b544-417d-9cd4-38d17452b08a None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:54.963607 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d624b7-b544-417d-9cd4-38d17452b08a None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:54.965425 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 94/184] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:01:54.971203 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26af575-7581-420f-8cba-f23ab4d4cebc None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:54.971669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26af575-7581-420f-8cba-f23ab4d4cebc None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:54.972049 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26af575-7581-420f-8cba-f23ab4d4cebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:54.994639 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-c26af575-7581-420f-8cba-f23ab4d4cebc None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:54.996098 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-c26af575-7581-420f-8cba-f23ab4d4cebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:55.008538 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 91/185] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 16:01:55.009488 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38164788-befa-4d78-9d5f-3bc1309352de None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:55.075728 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:55.080990 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 95/186] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:01:55.088798 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26af575-7581-420f-8cba-f23ab4d4cebc None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:55.137422 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38164788-befa-4d78-9d5f-3bc1309352de None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:55.165739 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:55.187627 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42a5dc-850c-4b5c-a9a4-c487f13b1bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:55.188445 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42a5dc-850c-4b5c-a9a4-c487f13b1bcb None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:55.188727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42a5dc-850c-4b5c-a9a4-c487f13b1bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:55.189089 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42a5dc-850c-4b5c-a9a4-c487f13b1bcb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:55.200081 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f42a5dc-850c-4b5c-a9a4-c487f13b1bcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:01:55.208356 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-4f42a5dc-850c-4b5c-a9a4-c487f13b1bcb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 16:01:55.208356 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:01:55.208356 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:01:55.208356 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:01:55.208356 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:01:55.208356 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:01:55.208356 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:01:55.208356 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:01:55.208356 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:01:55.208356 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:01:55.208356 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:01:55.208356 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:01:55.208356 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:01:55.208356 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:01:55.208356 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:01:55.208356 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:01:55.209795 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:01:55.209795 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:01:55.209795 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:01:55.209795 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:01:55.209795 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:01:55.209795 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:01:55.209795 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:01:55.209795 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:01:55.209795 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:01:55.209795 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:01:55.209795 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:01:55.209795 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:01:55.209795 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:01:55.209795 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:01:55.209795 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:01:55.209795 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:01:55.210600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:01:55.210600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:01:55.210600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:01:55.210600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:01:55.210600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:01:55.210600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:01:55.210600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:01:55.210600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:01:55.210600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:01:55.210600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:01:55.210600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 16:01:55.210600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:01:55.210600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 92/187] 10.222.0.41 () {56 vars in 1114 bytes} [Fri Jun 3 16:01:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:01:55.211256 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42a5dc-850c-4b5c-a9a4-c487f13b1bcb None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:55.229341 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26af575-7581-420f-8cba-f23ab4d4cebc None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:55.272917 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a29d37-c5f6-4dd6-a4de-b39f609647a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:55.273719 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a29d37-c5f6-4dd6-a4de-b39f609647a0 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:55.274014 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a29d37-c5f6-4dd6-a4de-b39f609647a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:55.274345 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a29d37-c5f6-4dd6-a4de-b39f609647a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:55.285117 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a29d37-c5f6-4dd6-a4de-b39f609647a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:01:55.293450 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-70a29d37-c5f6-4dd6-a4de-b39f609647a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:01:55.293450 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:01:55.293450 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:01:55.293450 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:01:55.293450 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:01:55.293450 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:01:55.293450 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:01:55.293450 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:01:55.293450 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:01:55.293450 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:01:55.293450 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:01:55.293450 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:01:55.293450 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:01:55.293450 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:01:55.293450 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:01:55.293450 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:01:55.294312 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:01:55.294312 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:01:55.294312 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:01:55.294312 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:01:55.294312 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:01:55.294312 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:01:55.294312 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:01:55.294312 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:01:55.294312 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:01:55.294312 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:01:55.294312 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:01:55.294312 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:01:55.294312 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:01:55.294312 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:01:55.294312 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:01:55.294312 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:01:55.295077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:01:55.295077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:01:55.295077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:01:55.295077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:01:55.295077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:01:55.295077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:01:55.295077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:01:55.295077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:01:55.295077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:01:55.295077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:01:55.295077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:01:55.295077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:01:55.299427 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 96/188] 10.222.0.41 () {56 vars in 1096 bytes} [Fri Jun 3 16:01:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:01:55.300285 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a29d37-c5f6-4dd6-a4de-b39f609647a0 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:55.324365 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42a5dc-850c-4b5c-a9a4-c487f13b1bcb None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:55.357348 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:55.360514 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ff4fb-2e2c-4f27-adb3-c4980a904b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:55.361361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ff4fb-2e2c-4f27-adb3-c4980a904b00 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:55.361891 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ff4fb-2e2c-4f27-adb3-c4980a904b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:55.361891 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ff4fb-2e2c-4f27-adb3-c4980a904b00 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:55.363093 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693ff4fb-2e2c-4f27-adb3-c4980a904b00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:55.364627 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693ff4fb-2e2c-4f27-adb3-c4980a904b00 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:55.370465 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 93/189] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:01:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:55.370465 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ff4fb-2e2c-4f27-adb3-c4980a904b00 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:55.410392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a29d37-c5f6-4dd6-a4de-b39f609647a0 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:55.423263 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:55.445322 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea660d5-c9f8-4ebc-afe3-46286a9e0705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:55.445941 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea660d5-c9f8-4ebc-afe3-46286a9e0705 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:55.446226 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea660d5-c9f8-4ebc-afe3-46286a9e0705 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:55.447095 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea660d5-c9f8-4ebc-afe3-46286a9e0705 None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:55.447271 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea660d5-c9f8-4ebc-afe3-46286a9e0705 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:55.447602 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea660d5-c9f8-4ebc-afe3-46286a9e0705 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:55.448995 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea660d5-c9f8-4ebc-afe3-46286a9e0705 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:55.457624 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 97/190] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:01:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:55.460804 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea660d5-c9f8-4ebc-afe3-46286a9e0705 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:55.478507 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ff4fb-2e2c-4f27-adb3-c4980a904b00 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:55.520488 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa9f1d-fee9-415c-be22-3ca171295f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:55.521716 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa9f1d-fee9-415c-be22-3ca171295f96 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:55.522249 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa9f1d-fee9-415c-be22-3ca171295f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:55.522539 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa9f1d-fee9-415c-be22-3ca171295f96 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:55.523724 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa9f1d-fee9-415c-be22-3ca171295f96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:55.524302 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa9f1d-fee9-415c-be22-3ca171295f96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:55.526419 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa9f1d-fee9-415c-be22-3ca171295f96 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:55.542057 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 94/191] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:01:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:55.542057 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa9f1d-fee9-415c-be22-3ca171295f96 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:55.564444 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea660d5-c9f8-4ebc-afe3-46286a9e0705 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:55.580576 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:55.599744 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565372c4-371a-4ab5-bfe1-545cac9ee37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:55.600415 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565372c4-371a-4ab5-bfe1-545cac9ee37e None admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:55.600704 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565372c4-371a-4ab5-bfe1-545cac9ee37e None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:55.600981 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565372c4-371a-4ab5-bfe1-545cac9ee37e None admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:55.601692 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565372c4-371a-4ab5-bfe1-545cac9ee37e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:55.603123 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565372c4-371a-4ab5-bfe1-545cac9ee37e None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:55.631409 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 98/192] 10.222.0.41 () {60 vars in 1132 bytes} [Fri Jun 3 16:01:55 2022] POST /identity/v3/users => generated 297 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:01:55.631409 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565372c4-371a-4ab5-bfe1-545cac9ee37e None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:55.660712 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa9f1d-fee9-415c-be22-3ca171295f96 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:55.675928 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:55.709917 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83217e-9536-4355-bff6-0b285041fdc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:55.710834 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83217e-9536-4355-bff6-0b285041fdc2 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:55.711119 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83217e-9536-4355-bff6-0b285041fdc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:55.711431 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83217e-9536-4355-bff6-0b285041fdc2 None admin] PATH_INFO: `/v3/users/0d298a0c04ac450587f90f28ad571c57` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:55.730779 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f83217e-9536-4355-bff6-0b285041fdc2 None admin] RBAC: Authorizing `identity:get_user(user_id=0d298a0c04ac450587f90f28ad571c57)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:55.732810 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f83217e-9536-4355-bff6-0b285041fdc2 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:55.739496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 95/193] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:01:55 2022] GET /identity/v3/users/0d298a0c04ac450587f90f28ad571c57 => generated 283 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:55.744376 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83217e-9536-4355-bff6-0b285041fdc2 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:55.764672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565372c4-371a-4ab5-bfe1-545cac9ee37e None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:55.812057 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1dc184-c9ee-47f6-8c03-1aa6b948dcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:55.813489 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1dc184-c9ee-47f6-8c03-1aa6b948dcfe None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:55.814789 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1dc184-c9ee-47f6-8c03-1aa6b948dcfe None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:55.815222 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1dc184-c9ee-47f6-8c03-1aa6b948dcfe None admin] PATH_INFO: `/v3/users/0d298a0c04ac450587f90f28ad571c57` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:55.817199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1dc184-c9ee-47f6-8c03-1aa6b948dcfe None admin] RBAC: Authorizing `identity:get_user(user_id=0d298a0c04ac450587f90f28ad571c57)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:55.819240 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1dc184-c9ee-47f6-8c03-1aa6b948dcfe None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:55.821135 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 99/194] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:01:55 2022] GET /identity/v3/users/0d298a0c04ac450587f90f28ad571c57 => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:55.827322 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1dc184-c9ee-47f6-8c03-1aa6b948dcfe None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:55.886052 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83217e-9536-4355-bff6-0b285041fdc2 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:55.888865 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:55.934190 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d752d45c-3ec9-43ac-8e8d-4de26f641acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:55.934871 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752d45c-3ec9-43ac-8e8d-4de26f641acf None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:55.935159 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752d45c-3ec9-43ac-8e8d-4de26f641acf None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:55.935462 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752d45c-3ec9-43ac-8e8d-4de26f641acf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:55.960157 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1dc184-c9ee-47f6-8c03-1aa6b948dcfe None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:55.975102 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:56.005097 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab210da-96dc-472a-acd1-078d5308c2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:56.005765 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab210da-96dc-472a-acd1-078d5308c2fb None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:56.006048 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab210da-96dc-472a-acd1-078d5308c2fb None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:56.006322 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab210da-96dc-472a-acd1-078d5308c2fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:56.026671 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d752d45c-3ec9-43ac-8e8d-4de26f641acf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:56.028078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d752d45c-3ec9-43ac-8e8d-4de26f641acf None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:56.137623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab210da-96dc-472a-acd1-078d5308c2fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:56.139115 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab210da-96dc-472a-acd1-078d5308c2fb None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:56.163929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 96/195] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:01:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 16:01:56.166629 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d752d45c-3ec9-43ac-8e8d-4de26f641acf None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:56.242740 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:56.265712 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:56.286379 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d752d45c-3ec9-43ac-8e8d-4de26f641acf None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:56.324592 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70eec0c-cccf-4540-9f4c-707d734ae99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:56.325803 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70eec0c-cccf-4540-9f4c-707d734ae99b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:56.326284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70eec0c-cccf-4540-9f4c-707d734ae99b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:56.326779 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70eec0c-cccf-4540-9f4c-707d734ae99b None admin] PATH_INFO: `/v3/projects/23d03cc631264a3faa207c1b3bded8e3` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:56.328203 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70eec0c-cccf-4540-9f4c-707d734ae99b None admin] RBAC: Authorizing `identity:get_project(project_id=23d03cc631264a3faa207c1b3bded8e3)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:56.329895 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70eec0c-cccf-4540-9f4c-707d734ae99b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:56.336738 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 97/196] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:01:56 2022] GET /identity/v3/projects/23d03cc631264a3faa207c1b3bded8e3 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:01:56.338157 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70eec0c-cccf-4540-9f4c-707d734ae99b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:56.391321 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 100/197] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:01:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 16:01:56.404054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab210da-96dc-472a-acd1-078d5308c2fb None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:56.451509 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70eec0c-cccf-4540-9f4c-707d734ae99b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:56.490001 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1608c35-679e-4072-9802-91def0a6b59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:56.491115 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1608c35-679e-4072-9802-91def0a6b59d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:56.491729 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1608c35-679e-4072-9802-91def0a6b59d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:56.492203 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1608c35-679e-4072-9802-91def0a6b59d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:56.493291 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1608c35-679e-4072-9802-91def0a6b59d None admin] RBAC: Adding query filter params (role.id=2248ac4d70c34e459b8942dec0d7656b, scope.project.id=23d03cc631264a3faa207c1b3bded8e3, user.id=0d298a0c04ac450587f90f28ad571c57) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:56.493802 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1608c35-679e-4072-9802-91def0a6b59d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:56.495501 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1608c35-679e-4072-9802-91def0a6b59d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:56.498795 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:56.502584 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 98/198] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:01:56 2022] GET /identity/v3/role_assignments?user.id=0d298a0c04ac450587f90f28ad571c57&scope.project.id=23d03cc631264a3faa207c1b3bded8e3&role.id=2248ac4d70c34e459b8942dec0d7656b => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:01:56.551942 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab210da-96dc-472a-acd1-078d5308c2fb None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:56.595144 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27788f8a-6591-484f-ad91-e3c5b914aa25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:56.596065 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27788f8a-6591-484f-ad91-e3c5b914aa25 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:56.596348 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27788f8a-6591-484f-ad91-e3c5b914aa25 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:56.596619 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27788f8a-6591-484f-ad91-e3c5b914aa25 None admin] PATH_INFO: `/v3/projects/23d03cc631264a3faa207c1b3bded8e3` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:56.600585 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27788f8a-6591-484f-ad91-e3c5b914aa25 None admin] RBAC: Authorizing `identity:get_project(project_id=23d03cc631264a3faa207c1b3bded8e3)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:56.602068 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27788f8a-6591-484f-ad91-e3c5b914aa25 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:56.603918 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 101/199] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:01:56 2022] GET /identity/v3/projects/23d03cc631264a3faa207c1b3bded8e3 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:01:56.610099 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1608c35-679e-4072-9802-91def0a6b59d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:56.726914 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1608c35-679e-4072-9802-91def0a6b59d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:56.765071 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f698bec-19d3-44c9-95bc-97243c3c90ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:56.766082 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f698bec-19d3-44c9-95bc-97243c3c90ca None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:56.766586 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f698bec-19d3-44c9-95bc-97243c3c90ca None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:56.767072 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f698bec-19d3-44c9-95bc-97243c3c90ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:56.768279 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f698bec-19d3-44c9-95bc-97243c3c90ca None admin] RBAC: Adding query filter params (role.id=75936a608b8149099ff54e317f3a6740, scope.project.id=23d03cc631264a3faa207c1b3bded8e3, user.id=0d298a0c04ac450587f90f28ad571c57) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:56.768810 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f698bec-19d3-44c9-95bc-97243c3c90ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:56.770502 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f698bec-19d3-44c9-95bc-97243c3c90ca None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:56.777750 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 99/200] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:01:56 2022] GET /identity/v3/role_assignments?user.id=0d298a0c04ac450587f90f28ad571c57&scope.project.id=23d03cc631264a3faa207c1b3bded8e3&role.id=75936a608b8149099ff54e317f3a6740 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:57.417170 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3863f6-36cc-4aae-9e36-d9bc5688c769 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:57.417673 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3863f6-36cc-4aae-9e36-d9bc5688c769 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:57.417746 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3863f6-36cc-4aae-9e36-d9bc5688c769 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:57.419253 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 102/201] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:01:57.425414 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e609bc-1382-45bf-b34f-4e7a4b64d8eb None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:57.425705 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e609bc-1382-45bf-b34f-4e7a4b64d8eb None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:57.425984 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e609bc-1382-45bf-b34f-4e7a4b64d8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:57.456619 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-14e609bc-1382-45bf-b34f-4e7a4b64d8eb None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:57.457868 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-14e609bc-1382-45bf-b34f-4e7a4b64d8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:57.541421 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:57.553192 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 100/202] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 16:01:57.584219 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3863f6-36cc-4aae-9e36-d9bc5688c769 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:57.598244 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:57.743922 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3863f6-36cc-4aae-9e36-d9bc5688c769 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:57.793922 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228826ad-3555-4fcf-bb52-43850db60bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:57.795123 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228826ad-3555-4fcf-bb52-43850db60bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:57.795631 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228826ad-3555-4fcf-bb52-43850db60bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:57.796114 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228826ad-3555-4fcf-bb52-43850db60bf5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:57.814024 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228826ad-3555-4fcf-bb52-43850db60bf5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:01:57.822707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-228826ad-3555-4fcf-bb52-43850db60bf5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 16:01:57.822707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:01:57.822707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:01:57.822707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:01:57.822707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:01:57.822707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:01:57.822707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:01:57.822707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:01:57.822707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:01:57.822707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:01:57.822707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:01:57.822707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:01:57.822707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:01:57.822707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:01:57.822707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:01:57.822707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:01:57.823593 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:01:57.823593 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:01:57.823593 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:01:57.823593 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:01:57.823593 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:01:57.823593 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:01:57.823593 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:01:57.823593 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:01:57.823593 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:01:57.823593 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:01:57.823593 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:01:57.823593 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:01:57.823593 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:01:57.823593 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:01:57.823593 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:01:57.823593 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:01:57.824375 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:01:57.824375 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:01:57.824375 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:01:57.824375 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:01:57.824375 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:01:57.824375 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:01:57.824375 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:01:57.824375 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:01:57.824375 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:01:57.824375 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:01:57.824375 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 16:01:57.824375 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:01:57.827007 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 103/203] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:01:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:01:57.832380 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e609bc-1382-45bf-b34f-4e7a4b64d8eb None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:57.961531 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e609bc-1382-45bf-b34f-4e7a4b64d8eb None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:58.002391 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c29146-a5f7-4c1c-8e0c-70378258f476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:58.003071 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c29146-a5f7-4c1c-8e0c-70378258f476 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:58.003377 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c29146-a5f7-4c1c-8e0c-70378258f476 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:58.003655 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c29146-a5f7-4c1c-8e0c-70378258f476 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:58.004408 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c29146-a5f7-4c1c-8e0c-70378258f476 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:58.004712 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c29146-a5f7-4c1c-8e0c-70378258f476 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:58.006704 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c29146-a5f7-4c1c-8e0c-70378258f476 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:58.015706 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 101/204] 10.222.0.41 () {56 vars in 1101 bytes} [Fri Jun 3 16:01:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:58.021149 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228826ad-3555-4fcf-bb52-43850db60bf5 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:58.160681 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:58.205810 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228826ad-3555-4fcf-bb52-43850db60bf5 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:58.269557 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7215d57e-c431-402f-8315-35ef4576fd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:58.270825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215d57e-c431-402f-8315-35ef4576fd00 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:58.271396 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215d57e-c431-402f-8315-35ef4576fd00 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:58.271872 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215d57e-c431-402f-8315-35ef4576fd00 None admin] PATH_INFO: `/v3/users/03685faa84b0451694258e03a8a8dc1e` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:58.276175 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7215d57e-c431-402f-8315-35ef4576fd00 None admin] RBAC: Authorizing `identity:get_user(user_id=03685faa84b0451694258e03a8a8dc1e)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:58.278160 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7215d57e-c431-402f-8315-35ef4576fd00 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:58.281812 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 104/205] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:01:58 2022] GET /identity/v3/users/03685faa84b0451694258e03a8a8dc1e => generated 289 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:58.293046 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c29146-a5f7-4c1c-8e0c-70378258f476 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:58.363908 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:58.422002 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c29146-a5f7-4c1c-8e0c-70378258f476 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:58.471842 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056bd809-df80-4667-a5d6-965191a72053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:58.472713 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056bd809-df80-4667-a5d6-965191a72053 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:58.472993 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056bd809-df80-4667-a5d6-965191a72053 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:58.473268 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056bd809-df80-4667-a5d6-965191a72053 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:58.567692 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056bd809-df80-4667-a5d6-965191a72053 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:58.573070 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056bd809-df80-4667-a5d6-965191a72053 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:58.638960 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5b10a-dd9a-484b-8fed-3c52e5583fec None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:58.639950 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5b10a-dd9a-484b-8fed-3c52e5583fec None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:58.640479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5b10a-dd9a-484b-8fed-3c52e5583fec None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:58.642046 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 105/206] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:01:58.646597 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:58.647923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2edb2b-37ca-47ae-a4a2-1f0992b34fc1 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:58.648715 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2edb2b-37ca-47ae-a4a2-1f0992b34fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:58.649194 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2edb2b-37ca-47ae-a4a2-1f0992b34fc1 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:58.651358 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 106/207] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:01:58.652720 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23120b7-46b8-4ee7-bfd9-233cf216fc4c None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:58.653229 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23120b7-46b8-4ee7-bfd9-233cf216fc4c None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:58.653699 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23120b7-46b8-4ee7-bfd9-233cf216fc4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:58.676445 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-b23120b7-46b8-4ee7-bfd9-233cf216fc4c None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:58.678493 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-b23120b7-46b8-4ee7-bfd9-233cf216fc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:58.693191 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 102/208] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:01:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 16:01:58.694709 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e065b74-70ce-40a3-b57c-7b55a9b7c2ef None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:58.695003 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e065b74-70ce-40a3-b57c-7b55a9b7c2ef None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:58.695283 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e065b74-70ce-40a3-b57c-7b55a9b7c2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:58.712833 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-8e065b74-70ce-40a3-b57c-7b55a9b7c2ef None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:58.714254 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8e065b74-70ce-40a3-b57c-7b55a9b7c2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:58.776187 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 107/209] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:01:58.780487 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23120b7-46b8-4ee7-bfd9-233cf216fc4c None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:58.813229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 103/210] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:01:58.814192 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e065b74-70ce-40a3-b57c-7b55a9b7c2ef None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:58.867268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:58.912352 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23120b7-46b8-4ee7-bfd9-233cf216fc4c None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:58.943201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:58.961780 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41fe317-c0dc-4f66-9d69-c6af9b566a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:58.962798 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41fe317-c0dc-4f66-9d69-c6af9b566a22 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:58.963072 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41fe317-c0dc-4f66-9d69-c6af9b566a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:58.963466 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41fe317-c0dc-4f66-9d69-c6af9b566a22 None admin] PATH_INFO: `/v3/projects/1d2895687ae2471682919de9b8fa4afb` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:58.964816 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41fe317-c0dc-4f66-9d69-c6af9b566a22 None admin] RBAC: Authorizing `identity:get_project(project_id=1d2895687ae2471682919de9b8fa4afb)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:58.965502 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e065b74-70ce-40a3-b57c-7b55a9b7c2ef None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:58.966412 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41fe317-c0dc-4f66-9d69-c6af9b566a22 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:58.969144 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 108/211] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:01:58 2022] GET /identity/v3/projects/1d2895687ae2471682919de9b8fa4afb => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:58.969144 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41fe317-c0dc-4f66-9d69-c6af9b566a22 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:59.004972 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5c2c2-62b3-4693-927a-0442af418a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:59.006711 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5c2c2-62b3-4693-927a-0442af418a07 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:59.007199 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5c2c2-62b3-4693-927a-0442af418a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:59.007752 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5c2c2-62b3-4693-927a-0442af418a07 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:59.018658 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5c2c2-62b3-4693-927a-0442af418a07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:01:59.026939 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-ebf5c2c2-62b3-4693-927a-0442af418a07 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 16:01:59.026939 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:01:59.026939 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:01:59.026939 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:01:59.026939 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:01:59.026939 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:01:59.026939 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:01:59.026939 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:01:59.026939 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:01:59.026939 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:01:59.026939 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:01:59.026939 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:01:59.026939 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:01:59.026939 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:01:59.026939 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:01:59.026939 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:01:59.027827 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:01:59.027827 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:01:59.027827 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:01:59.027827 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:01:59.027827 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:01:59.027827 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:01:59.027827 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:01:59.027827 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:01:59.027827 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:01:59.027827 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:01:59.027827 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:01:59.027827 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:01:59.027827 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:01:59.027827 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:01:59.027827 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:01:59.027827 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:01:59.028639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:01:59.028639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:01:59.028639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:01:59.028639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:01:59.028639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:01:59.028639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:01:59.028639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:01:59.028639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:01:59.028639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:01:59.028639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:01:59.028639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 16:01:59.028639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:01:59.031203 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 104/212] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:01:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:01:59.032186 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5c2c2-62b3-4693-927a-0442af418a07 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:59.073919 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41fe317-c0dc-4f66-9d69-c6af9b566a22 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:59.110546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419bf6f-6d91-459e-982f-6d686f5e7b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:59.111380 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419bf6f-6d91-459e-982f-6d686f5e7b84 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:59.111667 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419bf6f-6d91-459e-982f-6d686f5e7b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:59.111944 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419bf6f-6d91-459e-982f-6d686f5e7b84 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:59.113112 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d419bf6f-6d91-459e-982f-6d686f5e7b84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:59.115556 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d419bf6f-6d91-459e-982f-6d686f5e7b84 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:59.117698 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 109/213] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:01:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:59.118297 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419bf6f-6d91-459e-982f-6d686f5e7b84 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:59.123597 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:59.142224 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5c2c2-62b3-4693-927a-0442af418a07 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:59.175681 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f27862-7df9-4c6d-a56a-514c433d92a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:59.176331 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f27862-7df9-4c6d-a56a-514c433d92a5 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:59.176615 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f27862-7df9-4c6d-a56a-514c433d92a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:59.176890 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f27862-7df9-4c6d-a56a-514c433d92a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:59.178375 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f27862-7df9-4c6d-a56a-514c433d92a5 None admin] RBAC: Adding query filter params (role.id=23e883fa91ba4c4f8fa46a1414e9761a, scope.project.id=1d2895687ae2471682919de9b8fa4afb, user.id=03685faa84b0451694258e03a8a8dc1e) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:59.178693 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f27862-7df9-4c6d-a56a-514c433d92a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:59.180188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f27862-7df9-4c6d-a56a-514c433d92a5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:59.185993 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 105/214] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:01:59 2022] GET /identity/v3/role_assignments?user.id=03685faa84b0451694258e03a8a8dc1e&scope.project.id=1d2895687ae2471682919de9b8fa4afb&role.id=23e883fa91ba4c4f8fa46a1414e9761a => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:59.187389 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c489c9-92b8-42b2-bce5-1e04e9bce4b5 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:59.187682 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c489c9-92b8-42b2-bce5-1e04e9bce4b5 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:59.187963 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c489c9-92b8-42b2-bce5-1e04e9bce4b5 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:59.189333 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 106/215] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:01:59.194758 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c489c9-92b8-42b2-bce5-1e04e9bce4b5 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:59.211015 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:59.241362 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419bf6f-6d91-459e-982f-6d686f5e7b84 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:59.284022 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a07a96-42a8-4c35-adba-f9ed56d07519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:59.285058 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a07a96-42a8-4c35-adba-f9ed56d07519 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:59.285536 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a07a96-42a8-4c35-adba-f9ed56d07519 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:59.286152 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a07a96-42a8-4c35-adba-f9ed56d07519 None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:59.287856 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a07a96-42a8-4c35-adba-f9ed56d07519 None admin] RBAC: Adding query filter params (name=member) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:59.288422 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a07a96-42a8-4c35-adba-f9ed56d07519 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:59.290085 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a07a96-42a8-4c35-adba-f9ed56d07519 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:59.301707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 110/216] 10.222.0.41 () {56 vars in 1101 bytes} [Fri Jun 3 16:01:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:59.304082 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d49adb-0aba-4429-a4b2-597b348bda43 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:59.304566 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d49adb-0aba-4429-a4b2-597b348bda43 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:59.305060 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d49adb-0aba-4429-a4b2-597b348bda43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:59.324294 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-e2d49adb-0aba-4429-a4b2-597b348bda43 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:01:59.325867 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-e2d49adb-0aba-4429-a4b2-597b348bda43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:01:59.332079 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c489c9-92b8-42b2-bce5-1e04e9bce4b5 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:59.377680 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb8f514-8a79-4d11-a698-653fd0cf121b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:59.378759 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8f514-8a79-4d11-a698-653fd0cf121b None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:59.379278 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8f514-8a79-4d11-a698-653fd0cf121b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:59.379777 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8f514-8a79-4d11-a698-653fd0cf121b None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:59.381391 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb8f514-8a79-4d11-a698-653fd0cf121b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:59.383051 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb8f514-8a79-4d11-a698-653fd0cf121b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:59.409612 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 107/217] 10.222.0.41 () {60 vars in 1132 bytes} [Fri Jun 3 16:01:59 2022] POST /identity/v3/users => generated 297 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:01:59.410889 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb8f514-8a79-4d11-a698-653fd0cf121b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:59.416609 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:59.428626 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 111/218] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:01:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:01:59.453768 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d49adb-0aba-4429-a4b2-597b348bda43 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:59.510938 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:59.566704 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb8f514-8a79-4d11-a698-653fd0cf121b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:59.593386 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d49adb-0aba-4429-a4b2-597b348bda43 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:59.608496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e389227-826f-4221-8fbe-bac0978f906a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:59.609634 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e389227-826f-4221-8fbe-bac0978f906a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:59.610076 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e389227-826f-4221-8fbe-bac0978f906a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:59.610524 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e389227-826f-4221-8fbe-bac0978f906a None admin] PATH_INFO: `/v3/users/1f833b9b107746d89d7019d9034a5843` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:59.628194 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e389227-826f-4221-8fbe-bac0978f906a None admin] RBAC: Authorizing `identity:get_user(user_id=1f833b9b107746d89d7019d9034a5843)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:59.630037 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e389227-826f-4221-8fbe-bac0978f906a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:59.632633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 108/219] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:01:59 2022] GET /identity/v3/users/1f833b9b107746d89d7019d9034a5843 => generated 297 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:59.640003 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e389227-826f-4221-8fbe-bac0978f906a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:59.642693 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84cd574-27cc-450d-8ea2-b55d6327e3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:59.643561 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84cd574-27cc-450d-8ea2-b55d6327e3ed None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:59.643857 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84cd574-27cc-450d-8ea2-b55d6327e3ed None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:59.644137 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84cd574-27cc-450d-8ea2-b55d6327e3ed None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:59.660395 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84cd574-27cc-450d-8ea2-b55d6327e3ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:01:59.670923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-e84cd574-27cc-450d-8ea2-b55d6327e3ed None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 16:01:59.670923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:01:59.670923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:01:59.670923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:01:59.670923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:01:59.670923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:01:59.670923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:01:59.670923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:01:59.670923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:01:59.670923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:01:59.670923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:01:59.670923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:01:59.670923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:01:59.670923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:01:59.670923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:01:59.670923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:01:59.672044 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:01:59.672044 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:01:59.672044 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:01:59.672044 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:01:59.672044 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:01:59.672044 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:01:59.672044 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:01:59.672044 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:01:59.672044 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:01:59.672044 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:01:59.672044 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:01:59.672044 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:01:59.672044 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:01:59.672044 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:01:59.672044 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:01:59.672044 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:01:59.678123 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:01:59.678123 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:01:59.678123 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:01:59.678123 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:01:59.678123 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:01:59.678123 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:01:59.678123 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:01:59.678123 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:01:59.678123 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:01:59.678123 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:01:59.678123 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 16:01:59.678123 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:01:59.678123 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 112/220] 10.222.0.41 () {56 vars in 1114 bytes} [Fri Jun 3 16:01:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:01:59.680279 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84cd574-27cc-450d-8ea2-b55d6327e3ed None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:59.757126 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:59.775277 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e389227-826f-4221-8fbe-bac0978f906a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:59.808659 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:01:59.815061 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84cd574-27cc-450d-8ea2-b55d6327e3ed None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:01:59.823684 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc65365-0cd5-439f-82ad-0a2a421f2439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:59.824462 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc65365-0cd5-439f-82ad-0a2a421f2439 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:59.824572 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc65365-0cd5-439f-82ad-0a2a421f2439 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:59.824848 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc65365-0cd5-439f-82ad-0a2a421f2439 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:59.862160 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083882d5-1f53-44b7-bc31-774dc717857f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:01:59.862885 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083882d5-1f53-44b7-bc31-774dc717857f None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:59.863132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083882d5-1f53-44b7-bc31-774dc717857f None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:59.863455 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083882d5-1f53-44b7-bc31-774dc717857f None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:59.864218 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083882d5-1f53-44b7-bc31-774dc717857f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:01:59.864519 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083882d5-1f53-44b7-bc31-774dc717857f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:59.865923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083882d5-1f53-44b7-bc31-774dc717857f None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:01:59.879788 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 113/221] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:01:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:01:59.885276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302603ec-d303-40ff-ad66-7b8fd1ea74bb None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:01:59.885740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302603ec-d303-40ff-ad66-7b8fd1ea74bb None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:01:59.886323 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302603ec-d303-40ff-ad66-7b8fd1ea74bb None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:01:59.888700 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 114/222] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:01:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:01:59.888976 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302603ec-d303-40ff-ad66-7b8fd1ea74bb None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:01:59.922483 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc65365-0cd5-439f-82ad-0a2a421f2439 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:01:59.922483 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc65365-0cd5-439f-82ad-0a2a421f2439 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:00.003076 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302603ec-d303-40ff-ad66-7b8fd1ea74bb None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:00.026446 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:00.049952 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 109/223] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:01:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 16:02:00.050559 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdeb06d-64e5-4224-b2f9-a86b0c195ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:00.051240 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7372378-5b9d-4678-9a11-48dd79b90c0d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:00.051438 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdeb06d-64e5-4224-b2f9-a86b0c195ffe None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:00.051534 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7372378-5b9d-4678-9a11-48dd79b90c0d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:00.051713 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdeb06d-64e5-4224-b2f9-a86b0c195ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:00.051811 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7372378-5b9d-4678-9a11-48dd79b90c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:00.055377 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdeb06d-64e5-4224-b2f9-a86b0c195ffe None admin] PATH_INFO: `/v3/users/0d298a0c04ac450587f90f28ad571c57` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:00.055377 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdeb06d-64e5-4224-b2f9-a86b0c195ffe None admin] RBAC: Authorizing `identity:get_user(user_id=0d298a0c04ac450587f90f28ad571c57)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:00.055377 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdeb06d-64e5-4224-b2f9-a86b0c195ffe None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:00.057447 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 115/224] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:01:59 2022] GET /identity/v3/users/0d298a0c04ac450587f90f28ad571c57 => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:00.058297 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdeb06d-64e5-4224-b2f9-a86b0c195ffe None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:00.074523 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c7372378-5b9d-4678-9a11-48dd79b90c0d None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:00.075832 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c7372378-5b9d-4678-9a11-48dd79b90c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:00.101193 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:00.156916 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 110/225] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:00.157987 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7372378-5b9d-4678-9a11-48dd79b90c0d None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:00.182427 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdeb06d-64e5-4224-b2f9-a86b0c195ffe None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:00.232102 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c918ea8-6f50-4df9-9cce-ad04879dcf13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:00.233238 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c918ea8-6f50-4df9-9cce-ad04879dcf13 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:00.233548 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c918ea8-6f50-4df9-9cce-ad04879dcf13 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:00.234213 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c918ea8-6f50-4df9-9cce-ad04879dcf13 None admin] PATH_INFO: `/v3/projects/23d03cc631264a3faa207c1b3bded8e3` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:00.235731 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c918ea8-6f50-4df9-9cce-ad04879dcf13 None admin] RBAC: Authorizing `identity:get_project(project_id=23d03cc631264a3faa207c1b3bded8e3)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:00.238081 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c918ea8-6f50-4df9-9cce-ad04879dcf13 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:00.240380 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 116/226] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:02:00 2022] GET /identity/v3/projects/23d03cc631264a3faa207c1b3bded8e3 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:00.241460 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c918ea8-6f50-4df9-9cce-ad04879dcf13 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:00.283151 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7372378-5b9d-4678-9a11-48dd79b90c0d None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:00.301261 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:00.327327 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c9e81-e16f-4b29-a737-97ebe1836e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:00.328214 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c9e81-e16f-4b29-a737-97ebe1836e87 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:00.328498 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c9e81-e16f-4b29-a737-97ebe1836e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:00.328776 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c9e81-e16f-4b29-a737-97ebe1836e87 None admin] PATH_INFO: `/v3/projects/23d03cc631264a3faa207c1b3bded8e3` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:00.330095 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c9e81-e16f-4b29-a737-97ebe1836e87 None admin] RBAC: Authorizing `identity:get_project(project_id=23d03cc631264a3faa207c1b3bded8e3)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:00.331583 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c9e81-e16f-4b29-a737-97ebe1836e87 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:00.333449 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 111/227] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:02:00 2022] GET /identity/v3/projects/23d03cc631264a3faa207c1b3bded8e3 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:00.338329 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c9e81-e16f-4b29-a737-97ebe1836e87 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:00.364015 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c918ea8-6f50-4df9-9cce-ad04879dcf13 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:00.371539 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:00.406096 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b99472-393e-4662-a900-755fb90d96bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:00.407388 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b99472-393e-4662-a900-755fb90d96bb None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:00.407903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b99472-393e-4662-a900-755fb90d96bb None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:00.408370 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b99472-393e-4662-a900-755fb90d96bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:00.420107 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b99472-393e-4662-a900-755fb90d96bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:00.429276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-f1b99472-393e-4662-a900-755fb90d96bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:02:00.429276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:00.429276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:00.429276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:00.429276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:00.429276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:00.429276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:00.429276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:00.429276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:00.429276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:00.429276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:00.429276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:00.429276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:00.429276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:00.429276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:00.429276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:00.430132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:00.430132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:00.430132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:00.430132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:00.430132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:00.430132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:00.430132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:00.430132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:00.430132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:00.430132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:00.430132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:00.430132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:00.430132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:00.430132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:00.430132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:00.430132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:00.430919 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:00.430919 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:00.430919 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:00.430919 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:00.430919 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:00.430919 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:00.430919 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:00.430919 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:00.430919 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:00.430919 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:00.430919 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:02:00.430919 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:00.433787 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 117/228] 10.222.0.41 () {56 vars in 1096 bytes} [Fri Jun 3 16:02:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:00.434878 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b99472-393e-4662-a900-755fb90d96bb None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:00.465294 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c9e81-e16f-4b29-a737-97ebe1836e87 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:00.509489 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd10f25-496d-431a-aa2e-f0dcf3a07a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:00.510157 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd10f25-496d-431a-aa2e-f0dcf3a07a9a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:00.510443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd10f25-496d-431a-aa2e-f0dcf3a07a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:00.510723 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd10f25-496d-431a-aa2e-f0dcf3a07a9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:00.511878 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd10f25-496d-431a-aa2e-f0dcf3a07a9a None admin] RBAC: Adding query filter params (role.id=a8e4174e3572497f9284c709a119dcff, scope.project.id=23d03cc631264a3faa207c1b3bded8e3, user.id=1f833b9b107746d89d7019d9034a5843) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:00.512185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd10f25-496d-431a-aa2e-f0dcf3a07a9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:00.514155 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd10f25-496d-431a-aa2e-f0dcf3a07a9a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:00.521501 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 112/229] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:02:00 2022] GET /identity/v3/role_assignments?user.id=1f833b9b107746d89d7019d9034a5843&scope.project.id=23d03cc631264a3faa207c1b3bded8e3&role.id=a8e4174e3572497f9284c709a119dcff => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:00.522394 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd10f25-496d-431a-aa2e-f0dcf3a07a9a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:00.560335 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b99472-393e-4662-a900-755fb90d96bb None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:00.575452 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:00.604518 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f02334-12a9-4e83-9786-cc8f3ec89191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:00.605883 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f02334-12a9-4e83-9786-cc8f3ec89191 None admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:00.606360 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f02334-12a9-4e83-9786-cc8f3ec89191 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:00.606826 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f02334-12a9-4e83-9786-cc8f3ec89191 None admin] PATH_INFO: `/v3/projects/23d03cc631264a3faa207c1b3bded8e3/users/0d298a0c04ac450587f90f28ad571c57/roles/2248ac4d70c34e459b8942dec0d7656b` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:00.609591 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f02334-12a9-4e83-9786-cc8f3ec89191 None admin] RBAC: Authorizing `identity:create_grant(project_id=23d03cc631264a3faa207c1b3bded8e3, user_id=0d298a0c04ac450587f90f28ad571c57, role_id=2248ac4d70c34e459b8942dec0d7656b)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:00.611348 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f02334-12a9-4e83-9786-cc8f3ec89191 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:00.626182 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:00.654691 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 118/230] 10.222.0.41 () {58 vars in 1439 bytes} [Fri Jun 3 16:02:00 2022] PUT /identity/v3/projects/23d03cc631264a3faa207c1b3bded8e3/users/0d298a0c04ac450587f90f28ad571c57/roles/2248ac4d70c34e459b8942dec0d7656b => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:02:00.664139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd10f25-496d-431a-aa2e-f0dcf3a07a9a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:00.718667 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb9e82-b859-4cb1-aa8d-37e495633afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:00.719853 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb9e82-b859-4cb1-aa8d-37e495633afc None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:00.720368 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb9e82-b859-4cb1-aa8d-37e495633afc None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:00.720871 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb9e82-b859-4cb1-aa8d-37e495633afc None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:00.721887 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb9e82-b859-4cb1-aa8d-37e495633afc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:00.722497 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb9e82-b859-4cb1-aa8d-37e495633afc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:00.724395 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb9e82-b859-4cb1-aa8d-37e495633afc None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:00.736142 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 113/231] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:02:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:00.742474 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f02334-12a9-4e83-9786-cc8f3ec89191 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:00.892130 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f02334-12a9-4e83-9786-cc8f3ec89191 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:00.933697 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a2870f-9b7a-4fad-9907-887d16e10ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:00.934920 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2870f-9b7a-4fad-9907-887d16e10ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:00.935441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2870f-9b7a-4fad-9907-887d16e10ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:00.935986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2870f-9b7a-4fad-9907-887d16e10ab9 None admin] PATH_INFO: `/v3/users/0d298a0c04ac450587f90f28ad571c57` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:00.937422 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a2870f-9b7a-4fad-9907-887d16e10ab9 None admin] RBAC: Authorizing `identity:get_user(user_id=0d298a0c04ac450587f90f28ad571c57)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:00.939065 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a2870f-9b7a-4fad-9907-887d16e10ab9 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:00.941211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 119/232] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:02:00 2022] GET /identity/v3/users/0d298a0c04ac450587f90f28ad571c57 => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:00.946881 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb9e82-b859-4cb1-aa8d-37e495633afc None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:01.072809 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb9e82-b859-4cb1-aa8d-37e495633afc None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:01.119188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c6749f-bab0-41d4-88fb-e597ed9af419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:01.120090 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c6749f-bab0-41d4-88fb-e597ed9af419 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:01.120374 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c6749f-bab0-41d4-88fb-e597ed9af419 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:01.120648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c6749f-bab0-41d4-88fb-e597ed9af419 None admin] PATH_INFO: `/v3/projects/23d03cc631264a3faa207c1b3bded8e3` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:01.121855 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c6749f-bab0-41d4-88fb-e597ed9af419 None admin] RBAC: Authorizing `identity:get_project(project_id=23d03cc631264a3faa207c1b3bded8e3)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:01.123349 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c6749f-bab0-41d4-88fb-e597ed9af419 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:01.125362 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 114/233] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:02:00 2022] GET /identity/v3/projects/23d03cc631264a3faa207c1b3bded8e3 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:02:01.131035 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a2870f-9b7a-4fad-9907-887d16e10ab9 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:01.203860 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:01.284469 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a2870f-9b7a-4fad-9907-887d16e10ab9 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:01.326837 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b470df-f2c5-49d2-ad50-dd5cf77aff91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:01.327776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b470df-f2c5-49d2-ad50-dd5cf77aff91 None admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:01.328064 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b470df-f2c5-49d2-ad50-dd5cf77aff91 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:01.328345 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b470df-f2c5-49d2-ad50-dd5cf77aff91 None admin] PATH_INFO: `/v3/projects/23d03cc631264a3faa207c1b3bded8e3/users/0d298a0c04ac450587f90f28ad571c57/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:01.336653 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b470df-f2c5-49d2-ad50-dd5cf77aff91 None admin] RBAC: Authorizing `identity:create_grant(project_id=23d03cc631264a3faa207c1b3bded8e3, user_id=0d298a0c04ac450587f90f28ad571c57, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:01.338111 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b470df-f2c5-49d2-ad50-dd5cf77aff91 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:01.379967 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 120/234] 10.222.0.41 () {58 vars in 1439 bytes} [Fri Jun 3 16:02:01 2022] PUT /identity/v3/projects/23d03cc631264a3faa207c1b3bded8e3/users/0d298a0c04ac450587f90f28ad571c57/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:02:02.162044 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32a405-6b2a-43b9-8a33-1cac888d89ba None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:02.163058 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32a405-6b2a-43b9-8a33-1cac888d89ba None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:02.163563 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32a405-6b2a-43b9-8a33-1cac888d89ba None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:02.165397 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 115/235] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:02.171076 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afa0a0-57f6-4849-ad4a-48df10babe25 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:02.171916 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afa0a0-57f6-4849-ad4a-48df10babe25 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:02.172394 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afa0a0-57f6-4849-ad4a-48df10babe25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:02.196052 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-06afa0a0-57f6-4849-ad4a-48df10babe25 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:02.197570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-06afa0a0-57f6-4849-ad4a-48df10babe25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:02.291408 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 121/236] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:02.323991 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32a405-6b2a-43b9-8a33-1cac888d89ba None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:02.349145 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:02.438097 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25118ee6-7132-4ef2-a546-f22b147c5654 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:02.438295 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25118ee6-7132-4ef2-a546-f22b147c5654 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:02.438555 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25118ee6-7132-4ef2-a546-f22b147c5654 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:02.440198 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 122/237] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 16:02:02.445689 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592c989-e5ff-43f0-838b-d4aa05d9252e None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:02.445974 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592c989-e5ff-43f0-838b-d4aa05d9252e None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:02.446249 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592c989-e5ff-43f0-838b-d4aa05d9252e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:02.456550 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32a405-6b2a-43b9-8a33-1cac888d89ba None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:02.463808 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-1592c989-e5ff-43f0-838b-d4aa05d9252e None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:02.465079 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-1592c989-e5ff-43f0-838b-d4aa05d9252e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:02.495005 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:02.516596 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a17bc7-b4b0-4335-888e-856970f6c4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:02.517393 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a17bc7-b4b0-4335-888e-856970f6c4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:02.517676 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a17bc7-b4b0-4335-888e-856970f6c4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:02.517949 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a17bc7-b4b0-4335-888e-856970f6c4b3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:02.536840 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a17bc7-b4b0-4335-888e-856970f6c4b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:02.549169 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-c2a17bc7-b4b0-4335-888e-856970f6c4b3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 16:02:02.549169 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:02.549169 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:02.549169 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:02.549169 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:02.549169 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:02.549169 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:02.549169 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:02.549169 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:02.549169 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:02.549169 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:02.549169 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:02.549169 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:02.549169 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:02.549169 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:02.549169 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:02.550068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:02.550068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:02.550068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:02.550068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:02.550068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:02.550068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:02.550068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:02.550068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:02.550068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:02.550068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:02.550068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:02.550068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:02.550068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:02.550068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:02.550068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:02.550068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:02.550851 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:02.550851 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:02.550851 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:02.550851 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:02.550851 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:02.550851 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:02.550851 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:02.550851 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:02.550851 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:02.550851 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:02.550851 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 16:02:02.550851 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:02.550851 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 116/238] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:02:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:02.557230 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a17bc7-b4b0-4335-888e-856970f6c4b3 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:02.577311 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 123/239] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 16:02:02.635554 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592c989-e5ff-43f0-838b-d4aa05d9252e None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:02.680358 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:02.705273 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a17bc7-b4b0-4335-888e-856970f6c4b3 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:02.766777 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a759821-68ea-486b-ac99-89338eb83deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:02.767656 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a759821-68ea-486b-ac99-89338eb83deb None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:02.767946 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a759821-68ea-486b-ac99-89338eb83deb None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:02.768223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a759821-68ea-486b-ac99-89338eb83deb None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:02.768970 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a759821-68ea-486b-ac99-89338eb83deb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:02.769282 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a759821-68ea-486b-ac99-89338eb83deb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:02.771456 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a759821-68ea-486b-ac99-89338eb83deb None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:02.780059 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592c989-e5ff-43f0-838b-d4aa05d9252e None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:02.789837 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 117/240] 10.222.0.41 () {56 vars in 1101 bytes} [Fri Jun 3 16:02:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:02:02.795331 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a759821-68ea-486b-ac99-89338eb83deb None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:02.840506 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c45b80-3235-4201-b596-7ba839367d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:02.841659 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c45b80-3235-4201-b596-7ba839367d8a None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:02.843759 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c45b80-3235-4201-b596-7ba839367d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:02.844084 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c45b80-3235-4201-b596-7ba839367d8a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:02.845708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c45b80-3235-4201-b596-7ba839367d8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:02.846859 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c45b80-3235-4201-b596-7ba839367d8a None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:02.848677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 124/241] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:02:02.854790 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c45b80-3235-4201-b596-7ba839367d8a None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:02.913343 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:02.949931 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a759821-68ea-486b-ac99-89338eb83deb None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:02.992946 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c45b80-3235-4201-b596-7ba839367d8a None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:03.021668 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91bd44b-4d7c-4b1b-aae9-c0ae86ac62ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:03.022568 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91bd44b-4d7c-4b1b-aae9-c0ae86ac62ba None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:03.022853 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91bd44b-4d7c-4b1b-aae9-c0ae86ac62ba None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:03.023243 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91bd44b-4d7c-4b1b-aae9-c0ae86ac62ba None admin] PATH_INFO: `/v3/users/22050a6f911c4f699488be5e1bcae98e` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:03.038165 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b32f8-9391-4d8b-849c-27180bc15a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:03.038864 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b32f8-9391-4d8b-849c-27180bc15a69 None admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:03.039150 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b32f8-9391-4d8b-849c-27180bc15a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:03.039481 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b32f8-9391-4d8b-849c-27180bc15a69 None admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:03.040400 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b32f8-9391-4d8b-849c-27180bc15a69 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:03.041895 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b32f8-9391-4d8b-849c-27180bc15a69 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:03.046892 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:03.056529 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91bd44b-4d7c-4b1b-aae9-c0ae86ac62ba None admin] RBAC: Authorizing `identity:get_user(user_id=22050a6f911c4f699488be5e1bcae98e)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:03.058049 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91bd44b-4d7c-4b1b-aae9-c0ae86ac62ba None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:03.060876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 118/242] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:02:02 2022] GET /identity/v3/users/22050a6f911c4f699488be5e1bcae98e => generated 297 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:03.066287 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91bd44b-4d7c-4b1b-aae9-c0ae86ac62ba None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:03.083862 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 125/243] 10.222.0.41 () {60 vars in 1132 bytes} [Fri Jun 3 16:02:02 2022] POST /identity/v3/users => generated 293 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:02:03.202353 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91bd44b-4d7c-4b1b-aae9-c0ae86ac62ba None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:03.269735 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24367325-93a4-4343-adbe-d6f7ef429fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:03.270408 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24367325-93a4-4343-adbe-d6f7ef429fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:03.271214 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24367325-93a4-4343-adbe-d6f7ef429fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:03.271551 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24367325-93a4-4343-adbe-d6f7ef429fb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:03.358533 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:03.368024 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24367325-93a4-4343-adbe-d6f7ef429fb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:03.369596 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24367325-93a4-4343-adbe-d6f7ef429fb0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:03.482061 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 119/244] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:02:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 16:02:03.487537 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b32f8-9391-4d8b-849c-27180bc15a69 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:03.566458 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b79913-7c0d-42de-b53f-e3cf2bf29e1a None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:03.566643 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b79913-7c0d-42de-b53f-e3cf2bf29e1a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:03.567645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b79913-7c0d-42de-b53f-e3cf2bf29e1a None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:03.569818 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 120/245] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 16:02:03.575871 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd832c8-c1e3-4834-a37f-39332275f96e None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:03.576160 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd832c8-c1e3-4834-a37f-39332275f96e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:03.576437 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd832c8-c1e3-4834-a37f-39332275f96e None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:03.581139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 121/246] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 16:02:03.581139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e84adb-bffc-4b17-abc0-26939de4c3b7 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:03.581139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e84adb-bffc-4b17-abc0-26939de4c3b7 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:03.581139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e84adb-bffc-4b17-abc0-26939de4c3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:03.602428 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:03.603892 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-f9e84adb-bffc-4b17-abc0-26939de4c3b7 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:03.605375 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f9e84adb-bffc-4b17-abc0-26939de4c3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:03.623367 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b32f8-9391-4d8b-849c-27180bc15a69 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:03.660442 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7545ba7b-8a1c-4841-a34f-0dc630fe6978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:03.661843 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7545ba7b-8a1c-4841-a34f-0dc630fe6978 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:03.662147 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7545ba7b-8a1c-4841-a34f-0dc630fe6978 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:03.662424 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7545ba7b-8a1c-4841-a34f-0dc630fe6978 None admin] PATH_INFO: `/v3/projects/23d03cc631264a3faa207c1b3bded8e3` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:03.663753 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7545ba7b-8a1c-4841-a34f-0dc630fe6978 None admin] RBAC: Authorizing `identity:get_project(project_id=23d03cc631264a3faa207c1b3bded8e3)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:03.665766 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7545ba7b-8a1c-4841-a34f-0dc630fe6978 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:03.668544 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 126/247] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:02:03 2022] GET /identity/v3/projects/23d03cc631264a3faa207c1b3bded8e3 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:02:03.670369 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8ffed-e1e2-4bf4-b817-c9ff3919573d None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:03.671265 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8ffed-e1e2-4bf4-b817-c9ff3919573d None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:03.671792 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8ffed-e1e2-4bf4-b817-c9ff3919573d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:03.689662 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 122/248] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:03.690533 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e84adb-bffc-4b17-abc0-26939de4c3b7 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:03.694296 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-07b8ffed-e1e2-4bf4-b817-c9ff3919573d None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:03.695032 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-07b8ffed-e1e2-4bf4-b817-c9ff3919573d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:03.727633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:03.787521 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 127/249] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:03.788388 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b8ffed-e1e2-4bf4-b817-c9ff3919573d None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:03.828041 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e84adb-bffc-4b17-abc0-26939de4c3b7 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:03.872293 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93982fb-369a-498a-a42b-7161454ec1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:03.873507 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93982fb-369a-498a-a42b-7161454ec1d9 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:03.873792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93982fb-369a-498a-a42b-7161454ec1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:03.874068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93982fb-369a-498a-a42b-7161454ec1d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:03.874910 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93982fb-369a-498a-a42b-7161454ec1d9 None admin] RBAC: Adding query filter params (role.id=23e883fa91ba4c4f8fa46a1414e9761a, scope.project.id=23d03cc631264a3faa207c1b3bded8e3, user.id=22050a6f911c4f699488be5e1bcae98e) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:03.875224 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93982fb-369a-498a-a42b-7161454ec1d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:03.877190 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93982fb-369a-498a-a42b-7161454ec1d9 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:03.882298 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:03.888124 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 123/250] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:02:03 2022] GET /identity/v3/role_assignments?user.id=22050a6f911c4f699488be5e1bcae98e&scope.project.id=23d03cc631264a3faa207c1b3bded8e3&role.id=23e883fa91ba4c4f8fa46a1414e9761a => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:03.888808 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93982fb-369a-498a-a42b-7161454ec1d9 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:03.949590 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b8ffed-e1e2-4bf4-b817-c9ff3919573d None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:04.003798 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:04.017213 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93982fb-369a-498a-a42b-7161454ec1d9 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:04.040056 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a157dc-503c-403d-97db-74009cb5a429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:04.040873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a157dc-503c-403d-97db-74009cb5a429 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:04.041155 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a157dc-503c-403d-97db-74009cb5a429 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:04.041431 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a157dc-503c-403d-97db-74009cb5a429 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:04.057289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17d078-9800-44bb-a9ab-c3bff104361a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:04.058106 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17d078-9800-44bb-a9ab-c3bff104361a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:04.058449 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17d078-9800-44bb-a9ab-c3bff104361a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:04.058720 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17d078-9800-44bb-a9ab-c3bff104361a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:04.070652 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a157dc-503c-403d-97db-74009cb5a429 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:04.071899 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e17d078-9800-44bb-a9ab-c3bff104361a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:04.080284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-5e17d078-9800-44bb-a9ab-c3bff104361a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 16:02:04.080284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:04.080284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:04.080284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:04.080284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:04.080284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:04.080284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:04.080284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:04.080284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:04.080284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:04.080284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:04.080284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:04.080284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:04.080284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:04.080284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:04.080284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:04.081195 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:04.081195 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:04.081195 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:04.081195 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:04.081195 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:04.081195 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:04.081195 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:04.081195 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:04.081195 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:04.081195 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:04.081195 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:04.081195 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:04.081195 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:04.081195 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:04.081195 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:04.081195 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:04.084104 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:04.084104 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:04.084104 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:04.084104 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:04.084104 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:04.084104 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:04.084104 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:04.084104 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:04.084104 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:04.084104 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:04.084104 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 16:02:04.084104 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:04.084104 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 124/251] 10.222.0.41 () {56 vars in 1114 bytes} [Fri Jun 3 16:02:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:04.085756 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-c4a157dc-503c-403d-97db-74009cb5a429 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 16:02:04.085756 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:04.085756 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:04.085756 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:04.085756 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:04.085756 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:04.085756 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:04.085756 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:04.085756 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:04.085756 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:04.085756 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:04.085756 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:04.085756 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:04.085756 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:04.085756 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:04.085756 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:04.090381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:04.090381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:04.090381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:04.090381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:04.090381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:04.090381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:04.090381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:04.090381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:04.090381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:04.090381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:04.090381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:04.090381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:04.090381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:04.090381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:04.090381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:04.090381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:04.092494 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:04.092494 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:04.092494 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:04.092494 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:04.092494 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:04.092494 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:04.092494 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:04.092494 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:04.092494 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:04.092494 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:04.092494 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 16:02:04.092494 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:04.092494 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 128/252] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:02:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:04.093825 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17d078-9800-44bb-a9ab-c3bff104361a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:04.093917 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a157dc-503c-403d-97db-74009cb5a429 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:04.210409 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17d078-9800-44bb-a9ab-c3bff104361a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:04.253610 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7dded2-d85e-4901-8b7c-8380b0b67f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:04.254975 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7dded2-d85e-4901-8b7c-8380b0b67f99 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:04.256013 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7dded2-d85e-4901-8b7c-8380b0b67f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:04.256431 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7dded2-d85e-4901-8b7c-8380b0b67f99 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:04.257324 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7dded2-d85e-4901-8b7c-8380b0b67f99 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:04.257773 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7dded2-d85e-4901-8b7c-8380b0b67f99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:04.259313 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7dded2-d85e-4901-8b7c-8380b0b67f99 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:04.269714 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 125/253] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:02:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:04.275318 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7dded2-d85e-4901-8b7c-8380b0b67f99 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:04.292831 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:04.352941 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a157dc-503c-403d-97db-74009cb5a429 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:04.390418 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e12907-f016-4253-b8b2-62c8bb567154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:04.391390 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e12907-f016-4253-b8b2-62c8bb567154 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:04.391834 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e12907-f016-4253-b8b2-62c8bb567154 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:04.392262 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e12907-f016-4253-b8b2-62c8bb567154 None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:04.393154 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e12907-f016-4253-b8b2-62c8bb567154 None admin] RBAC: Adding query filter params (name=member) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:04.393605 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e12907-f016-4253-b8b2-62c8bb567154 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:04.395132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e12907-f016-4253-b8b2-62c8bb567154 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:04.404649 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 129/254] 10.222.0.41 () {56 vars in 1101 bytes} [Fri Jun 3 16:02:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:02:04.406164 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766fabf-6071-43d3-8685-9810f18ac20d None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:04.406612 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766fabf-6071-43d3-8685-9810f18ac20d None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:04.407080 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766fabf-6071-43d3-8685-9810f18ac20d None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:04.408485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 130/255] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:04.415269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e74be-70f7-4944-bf5d-15f610786949 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:04.415751 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e74be-70f7-4944-bf5d-15f610786949 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:04.416165 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e74be-70f7-4944-bf5d-15f610786949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:04.439705 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:04.444322 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-f93e74be-70f7-4944-bf5d-15f610786949 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:04.448426 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-f93e74be-70f7-4944-bf5d-15f610786949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:04.470843 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7dded2-d85e-4901-8b7c-8380b0b67f99 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:04.535229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e244e5-e9c7-405f-9f98-b34affce7ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:04.536132 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e244e5-e9c7-405f-9f98-b34affce7ecb None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:04.536940 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e244e5-e9c7-405f-9f98-b34affce7ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:04.537217 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e244e5-e9c7-405f-9f98-b34affce7ecb None admin] PATH_INFO: `/v3/users/0d298a0c04ac450587f90f28ad571c57` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:04.538513 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e244e5-e9c7-405f-9f98-b34affce7ecb None admin] RBAC: Authorizing `identity:get_user(user_id=0d298a0c04ac450587f90f28ad571c57)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:04.540096 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e244e5-e9c7-405f-9f98-b34affce7ecb None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:04.547796 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 126/256] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:02:04 2022] GET /identity/v3/users/0d298a0c04ac450587f90f28ad571c57 => generated 283 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:04.552358 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e244e5-e9c7-405f-9f98-b34affce7ecb None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:04.561711 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 131/257] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:04.563219 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e74be-70f7-4944-bf5d-15f610786949 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:04.658756 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:04.677424 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e74be-70f7-4944-bf5d-15f610786949 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:04.697505 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e244e5-e9c7-405f-9f98-b34affce7ecb None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:04.713767 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1715388-4d01-49de-979f-e477353c65a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:04.714680 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1715388-4d01-49de-979f-e477353c65a0 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:04.715102 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1715388-4d01-49de-979f-e477353c65a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:04.715546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1715388-4d01-49de-979f-e477353c65a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:04.718323 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:04.782367 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e353c3-4e3f-4b15-a31c-3fbe8fd5e569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:04.783582 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e353c3-4e3f-4b15-a31c-3fbe8fd5e569 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:04.783582 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e353c3-4e3f-4b15-a31c-3fbe8fd5e569 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:04.783849 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e353c3-4e3f-4b15-a31c-3fbe8fd5e569 None admin] PATH_INFO: `/v3/users/1f833b9b107746d89d7019d9034a5843` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:04.785024 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e353c3-4e3f-4b15-a31c-3fbe8fd5e569 None admin] RBAC: Authorizing `identity:get_user(user_id=1f833b9b107746d89d7019d9034a5843)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:04.786157 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1715388-4d01-49de-979f-e477353c65a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:04.786536 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e353c3-4e3f-4b15-a31c-3fbe8fd5e569 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:04.788093 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1715388-4d01-49de-979f-e477353c65a0 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:04.788200 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 127/258] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:02:04 2022] GET /identity/v3/users/1f833b9b107746d89d7019d9034a5843 => generated 297 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:04.793185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e353c3-4e3f-4b15-a31c-3fbe8fd5e569 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:04.892776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 132/259] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:02:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 16:02:04.894060 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1715388-4d01-49de-979f-e477353c65a0 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:04.930120 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:05.003857 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1715388-4d01-49de-979f-e477353c65a0 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:05.042276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47fdf8b-a1e9-4eff-8686-035d2895005e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:05.043514 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47fdf8b-a1e9-4eff-8686-035d2895005e None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:05.043997 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47fdf8b-a1e9-4eff-8686-035d2895005e None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:05.044490 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47fdf8b-a1e9-4eff-8686-035d2895005e None admin] PATH_INFO: `/v3/projects/23d03cc631264a3faa207c1b3bded8e3` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:05.045868 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47fdf8b-a1e9-4eff-8686-035d2895005e None admin] RBAC: Authorizing `identity:get_project(project_id=23d03cc631264a3faa207c1b3bded8e3)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:05.048308 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47fdf8b-a1e9-4eff-8686-035d2895005e None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:05.050504 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 133/260] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:02:04 2022] GET /identity/v3/projects/23d03cc631264a3faa207c1b3bded8e3 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:05.051491 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47fdf8b-a1e9-4eff-8686-035d2895005e None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:05.161339 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47fdf8b-a1e9-4eff-8686-035d2895005e None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:05.165213 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e353c3-4e3f-4b15-a31c-3fbe8fd5e569 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:05.170596 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:05.201522 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a027e-b8e9-4193-a5b4-1b00219b57d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:05.202452 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a027e-b8e9-4193-a5b4-1b00219b57d6 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:05.202831 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a027e-b8e9-4193-a5b4-1b00219b57d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:05.203202 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a027e-b8e9-4193-a5b4-1b00219b57d6 None admin] PATH_INFO: `/v3/projects/23d03cc631264a3faa207c1b3bded8e3` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:05.204763 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a027e-b8e9-4193-a5b4-1b00219b57d6 None admin] RBAC: Authorizing `identity:get_project(project_id=23d03cc631264a3faa207c1b3bded8e3)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:05.206291 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a027e-b8e9-4193-a5b4-1b00219b57d6 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:05.209050 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 134/261] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:02:05 2022] GET /identity/v3/projects/23d03cc631264a3faa207c1b3bded8e3 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:05.209755 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a027e-b8e9-4193-a5b4-1b00219b57d6 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:05.263582 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b40fe-9a25-49fd-9048-4258e7993048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:05.265779 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b40fe-9a25-49fd-9048-4258e7993048 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:05.266269 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b40fe-9a25-49fd-9048-4258e7993048 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:05.266772 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b40fe-9a25-49fd-9048-4258e7993048 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:05.278988 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04b40fe-9a25-49fd-9048-4258e7993048 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:05.287719 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-e04b40fe-9a25-49fd-9048-4258e7993048 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:02:05.287719 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:05.287719 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:05.287719 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:05.287719 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:05.287719 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:05.287719 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:05.287719 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:05.287719 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:05.287719 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:05.287719 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:05.287719 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:05.287719 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:05.287719 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:05.287719 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:05.287719 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:05.288592 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:05.288592 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:05.288592 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:05.288592 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:05.288592 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:05.288592 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:05.288592 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:05.288592 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:05.288592 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:05.288592 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:05.288592 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:05.288592 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:05.288592 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:05.288592 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:05.288592 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:05.288592 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:05.289372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:05.289372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:05.289372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:05.289372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:05.289372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:05.289372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:05.289372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:05.289372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:05.289372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:05.289372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:05.289372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:02:05.289372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:05.291703 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 128/262] 10.222.0.41 () {56 vars in 1096 bytes} [Fri Jun 3 16:02:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 503 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:05.293197 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b40fe-9a25-49fd-9048-4258e7993048 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:05.306525 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:05.408215 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b40fe-9a25-49fd-9048-4258e7993048 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:05.445424 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930f55fd-1a73-46ed-8d20-c1253af3cf02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:05.446462 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f55fd-1a73-46ed-8d20-c1253af3cf02 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:05.446966 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f55fd-1a73-46ed-8d20-c1253af3cf02 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:05.447481 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f55fd-1a73-46ed-8d20-c1253af3cf02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:05.448509 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930f55fd-1a73-46ed-8d20-c1253af3cf02 None admin] RBAC: Adding query filter params (role.id=2248ac4d70c34e459b8942dec0d7656b, scope.project.id=23d03cc631264a3faa207c1b3bded8e3, user.id=0d298a0c04ac450587f90f28ad571c57) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:05.449038 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930f55fd-1a73-46ed-8d20-c1253af3cf02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:05.451190 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930f55fd-1a73-46ed-8d20-c1253af3cf02 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:05.458295 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 129/263] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:02:05 2022] GET /identity/v3/role_assignments?user.id=0d298a0c04ac450587f90f28ad571c57&scope.project.id=23d03cc631264a3faa207c1b3bded8e3&role.id=2248ac4d70c34e459b8942dec0d7656b => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:05.460190 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930f55fd-1a73-46ed-8d20-c1253af3cf02 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:05.482061 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a027e-b8e9-4193-a5b4-1b00219b57d6 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:05.546236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:05.581983 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930f55fd-1a73-46ed-8d20-c1253af3cf02 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:05.588661 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9223f-f35a-409f-9bac-fadbae74e757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:05.589591 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9223f-f35a-409f-9bac-fadbae74e757 None admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:05.589878 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9223f-f35a-409f-9bac-fadbae74e757 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:05.590154 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9223f-f35a-409f-9bac-fadbae74e757 None admin] PATH_INFO: `/v3/projects/23d03cc631264a3faa207c1b3bded8e3/users/1f833b9b107746d89d7019d9034a5843/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:05.600453 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a9223f-f35a-409f-9bac-fadbae74e757 None admin] RBAC: Authorizing `identity:create_grant(project_id=23d03cc631264a3faa207c1b3bded8e3, user_id=1f833b9b107746d89d7019d9034a5843, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:05.601938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a9223f-f35a-409f-9bac-fadbae74e757 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:05.623169 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27786e7e-2114-4880-b4a2-d72bc10ad893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:05.624280 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27786e7e-2114-4880-b4a2-d72bc10ad893 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:05.624817 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27786e7e-2114-4880-b4a2-d72bc10ad893 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:05.625327 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27786e7e-2114-4880-b4a2-d72bc10ad893 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:05.626314 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27786e7e-2114-4880-b4a2-d72bc10ad893 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:05.626869 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27786e7e-2114-4880-b4a2-d72bc10ad893 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:05.628623 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27786e7e-2114-4880-b4a2-d72bc10ad893 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:05.640275 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 130/264] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:02:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:05.643688 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 135/265] 10.222.0.41 () {58 vars in 1439 bytes} [Fri Jun 3 16:02:05 2022] PUT /identity/v3/projects/23d03cc631264a3faa207c1b3bded8e3/users/1f833b9b107746d89d7019d9034a5843/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:02:05.652609 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27786e7e-2114-4880-b4a2-d72bc10ad893 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:05.804325 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27786e7e-2114-4880-b4a2-d72bc10ad893 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:05.854712 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a5cdc-2077-401a-995e-20f7e17f15c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:05.856078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a5cdc-2077-401a-995e-20f7e17f15c4 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:05.856563 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a5cdc-2077-401a-995e-20f7e17f15c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:05.857056 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a5cdc-2077-401a-995e-20f7e17f15c4 None admin] PATH_INFO: `/v3/users/0d298a0c04ac450587f90f28ad571c57` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:05.858639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a5cdc-2077-401a-995e-20f7e17f15c4 None admin] RBAC: Authorizing `identity:get_user(user_id=0d298a0c04ac450587f90f28ad571c57)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:05.860538 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a5cdc-2077-401a-995e-20f7e17f15c4 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:05.862557 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 131/266] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:02:05 2022] GET /identity/v3/users/0d298a0c04ac450587f90f28ad571c57 => generated 283 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:05.868450 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9223f-f35a-409f-9bac-fadbae74e757 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:05.903722 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:05.985002 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9223f-f35a-409f-9bac-fadbae74e757 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:06.025660 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918c5200-893f-40be-9aa1-de15ccc84c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:06.026314 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c5200-893f-40be-9aa1-de15ccc84c35 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:06.026595 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c5200-893f-40be-9aa1-de15ccc84c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:06.026869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c5200-893f-40be-9aa1-de15ccc84c35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:06.104432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918c5200-893f-40be-9aa1-de15ccc84c35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:06.105918 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918c5200-893f-40be-9aa1-de15ccc84c35 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:06.142744 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:06.240599 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 136/267] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:02:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 16:02:06.246106 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a5cdc-2077-401a-995e-20f7e17f15c4 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:06.255793 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:06.358152 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a5cdc-2077-401a-995e-20f7e17f15c4 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:06.374850 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364f992-bbe7-4fd7-b521-15d940a56534 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:06.375149 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364f992-bbe7-4fd7-b521-15d940a56534 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:06.375479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364f992-bbe7-4fd7-b521-15d940a56534 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:06.377008 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 137/268] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 16:02:06.382607 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4cedd-0c10-4147-968d-3f83438b1239 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:06.382900 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4cedd-0c10-4147-968d-3f83438b1239 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:06.383529 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4cedd-0c10-4147-968d-3f83438b1239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:06.402400 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-00b4cedd-0c10-4147-968d-3f83438b1239 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:06.404905 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-00b4cedd-0c10-4147-968d-3f83438b1239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:06.419854 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a407c5-69fc-445f-b67f-6add81371ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:06.420727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a407c5-69fc-445f-b67f-6add81371ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:06.421010 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a407c5-69fc-445f-b67f-6add81371ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:06.421285 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a407c5-69fc-445f-b67f-6add81371ce5 None admin] PATH_INFO: `/v3/projects/23d03cc631264a3faa207c1b3bded8e3` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:06.424359 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a407c5-69fc-445f-b67f-6add81371ce5 None admin] RBAC: Authorizing `identity:get_project(project_id=23d03cc631264a3faa207c1b3bded8e3)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:06.425813 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a407c5-69fc-445f-b67f-6add81371ce5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:06.427613 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 132/269] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:02:06 2022] GET /identity/v3/projects/23d03cc631264a3faa207c1b3bded8e3 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:06.434923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a407c5-69fc-445f-b67f-6add81371ce5 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:06.531650 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:06.539521 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 138/270] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:06.540975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae3e94-f085-4e21-8b53-052e7b97d279 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:06.541265 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae3e94-f085-4e21-8b53-052e7b97d279 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:06.541543 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae3e94-f085-4e21-8b53-052e7b97d279 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:06.547191 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 139/271] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:06.548368 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcbd3e-9f8b-44f5-9312-5b8758f800e9 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:06.548656 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcbd3e-9f8b-44f5-9312-5b8758f800e9 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:06.548925 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcbd3e-9f8b-44f5-9312-5b8758f800e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:06.567608 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-98bcbd3e-9f8b-44f5-9312-5b8758f800e9 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:06.569384 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-98bcbd3e-9f8b-44f5-9312-5b8758f800e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:06.578875 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a407c5-69fc-445f-b67f-6add81371ce5 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:06.629579 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5759b-fffd-44c5-a20d-4ccc683e4161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:06.630189 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5759b-fffd-44c5-a20d-4ccc683e4161 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:06.630474 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5759b-fffd-44c5-a20d-4ccc683e4161 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:06.630751 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5759b-fffd-44c5-a20d-4ccc683e4161 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:06.631726 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec5759b-fffd-44c5-a20d-4ccc683e4161 None admin] RBAC: Adding query filter params (role.id=75936a608b8149099ff54e317f3a6740, scope.project.id=23d03cc631264a3faa207c1b3bded8e3, user.id=0d298a0c04ac450587f90f28ad571c57) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:06.632045 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec5759b-fffd-44c5-a20d-4ccc683e4161 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:06.633947 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:06.634667 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec5759b-fffd-44c5-a20d-4ccc683e4161 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:06.641030 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 133/272] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:02:06 2022] GET /identity/v3/role_assignments?user.id=0d298a0c04ac450587f90f28ad571c57&scope.project.id=23d03cc631264a3faa207c1b3bded8e3&role.id=75936a608b8149099ff54e317f3a6740 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:02:06.641910 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5759b-fffd-44c5-a20d-4ccc683e4161 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:06.647307 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 140/273] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 16:02:06.672602 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bcbd3e-9f8b-44f5-9312-5b8758f800e9 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:06.773328 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5759b-fffd-44c5-a20d-4ccc683e4161 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:06.807769 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bcbd3e-9f8b-44f5-9312-5b8758f800e9 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:06.848395 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b82e97f-91f8-47bc-a61b-f87468a26ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:06.849448 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82e97f-91f8-47bc-a61b-f87468a26ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:06.849729 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82e97f-91f8-47bc-a61b-f87468a26ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:06.850004 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82e97f-91f8-47bc-a61b-f87468a26ca8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:06.851252 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b82e97f-91f8-47bc-a61b-f87468a26ca8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:06.852945 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b82e97f-91f8-47bc-a61b-f87468a26ca8 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:06.854965 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 141/274] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:06.860607 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b82e97f-91f8-47bc-a61b-f87468a26ca8 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:06.913906 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334482ed-9889-47f4-b80c-e7bde7fdf49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:06.914994 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334482ed-9889-47f4-b80c-e7bde7fdf49a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:06.915283 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334482ed-9889-47f4-b80c-e7bde7fdf49a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:06.915627 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334482ed-9889-47f4-b80c-e7bde7fdf49a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:06.918276 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:06.934308 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334482ed-9889-47f4-b80c-e7bde7fdf49a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:06.939638 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:06.945718 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-334482ed-9889-47f4-b80c-e7bde7fdf49a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 16:02:06.945718 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:06.945718 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:06.945718 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:06.945718 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:06.945718 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:06.945718 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:06.945718 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:06.945718 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:06.945718 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:06.945718 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:06.945718 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:06.945718 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:06.945718 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:06.945718 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:06.945718 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:06.946632 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:06.946632 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:06.946632 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:06.946632 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:06.946632 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:06.946632 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:06.946632 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:06.946632 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:06.946632 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:06.946632 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:06.946632 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:06.946632 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:06.946632 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:06.946632 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:06.946632 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:06.946632 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:06.947452 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:06.947452 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:06.947452 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:06.947452 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:06.947452 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:06.947452 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:06.947452 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:06.947452 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:06.947452 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:06.947452 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:06.947452 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 16:02:06.947452 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:06.951030 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 134/275] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:02:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:06.954594 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334482ed-9889-47f4-b80c-e7bde7fdf49a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:06.992598 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b82e97f-91f8-47bc-a61b-f87468a26ca8 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:07.041789 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fb355c-2455-4af2-b993-76325c75c489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:07.042734 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb355c-2455-4af2-b993-76325c75c489 None admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:07.043023 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb355c-2455-4af2-b993-76325c75c489 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:07.043313 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb355c-2455-4af2-b993-76325c75c489 None admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:07.044699 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fb355c-2455-4af2-b993-76325c75c489 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:07.046632 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fb355c-2455-4af2-b993-76325c75c489 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:07.074649 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 142/276] 10.222.0.41 () {60 vars in 1132 bytes} [Fri Jun 3 16:02:06 2022] POST /identity/v3/users => generated 293 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:02:07.097257 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334482ed-9889-47f4-b80c-e7bde7fdf49a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:07.110537 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce7ed9-61fa-4187-9873-d9aa00c192b4 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:07.110861 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce7ed9-61fa-4187-9873-d9aa00c192b4 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:07.111093 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce7ed9-61fa-4187-9873-d9aa00c192b4 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:07.112950 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 143/277] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 16:02:07.128891 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90d791-7b05-42b0-980b-0c53324537f7 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:07.129205 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90d791-7b05-42b0-980b-0c53324537f7 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:07.129451 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90d791-7b05-42b0-980b-0c53324537f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:07.144559 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176fb39e-2515-4860-b780-5d7822d0812b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:07.146160 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176fb39e-2515-4860-b780-5d7822d0812b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:07.146707 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176fb39e-2515-4860-b780-5d7822d0812b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:07.147183 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176fb39e-2515-4860-b780-5d7822d0812b None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:07.148187 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176fb39e-2515-4860-b780-5d7822d0812b None admin] RBAC: Adding query filter params (name=member) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:07.148726 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176fb39e-2515-4860-b780-5d7822d0812b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:07.150368 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176fb39e-2515-4860-b780-5d7822d0812b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:07.154712 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-ca90d791-7b05-42b0-980b-0c53324537f7 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:07.155986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-ca90d791-7b05-42b0-980b-0c53324537f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:07.162038 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 135/278] 10.222.0.41 () {56 vars in 1101 bytes} [Fri Jun 3 16:02:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:02:07.168002 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176fb39e-2515-4860-b780-5d7822d0812b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:07.245003 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:07.253559 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 144/279] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:07.276992 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90d791-7b05-42b0-980b-0c53324537f7 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:07.308184 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176fb39e-2515-4860-b780-5d7822d0812b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:07.335693 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:07.359119 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3df2e-195e-4f8e-92da-ef458321ea94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:07.360042 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3df2e-195e-4f8e-92da-ef458321ea94 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:07.360322 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3df2e-195e-4f8e-92da-ef458321ea94 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:07.361194 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3df2e-195e-4f8e-92da-ef458321ea94 None admin] PATH_INFO: `/v3/users/a59e517545d741f1b8438c603c29fce2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:07.376632 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd3df2e-195e-4f8e-92da-ef458321ea94 None admin] RBAC: Authorizing `identity:get_user(user_id=a59e517545d741f1b8438c603c29fce2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:07.378376 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd3df2e-195e-4f8e-92da-ef458321ea94 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:07.380332 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 136/280] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:02:07 2022] GET /identity/v3/users/a59e517545d741f1b8438c603c29fce2 => generated 293 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:02:07.387517 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3df2e-195e-4f8e-92da-ef458321ea94 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:07.419718 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90d791-7b05-42b0-980b-0c53324537f7 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:07.464941 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de48f38-f4b2-443f-9138-c6c5c6a08d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:07.465812 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de48f38-f4b2-443f-9138-c6c5c6a08d3e None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:07.466099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de48f38-f4b2-443f-9138-c6c5c6a08d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:07.466413 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de48f38-f4b2-443f-9138-c6c5c6a08d3e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:07.481135 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de48f38-f4b2-443f-9138-c6c5c6a08d3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:07.490062 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-8de48f38-f4b2-443f-9138-c6c5c6a08d3e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 16:02:07.490062 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:07.490062 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:07.490062 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:07.490062 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:07.490062 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:07.490062 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:07.490062 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:07.490062 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:07.490062 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:07.490062 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:07.490062 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:07.490062 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:07.490062 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:07.490062 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:07.490062 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:07.491199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:07.491199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:07.491199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:07.491199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:07.491199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:07.491199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:07.491199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:07.491199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:07.491199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:07.491199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:07.491199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:07.491199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:07.491199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:07.491199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:07.491199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:07.491199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:07.492205 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:07.492205 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:07.492205 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:07.492205 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:07.492205 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:07.492205 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:07.492205 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:07.492205 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:07.492205 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:07.492205 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:07.492205 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 16:02:07.492205 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:07.494239 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 145/281] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:02:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:02:07.501046 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de48f38-f4b2-443f-9138-c6c5c6a08d3e None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:07.509986 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3df2e-195e-4f8e-92da-ef458321ea94 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:07.529374 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:07.563247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d3975-7ef1-49d2-bfd7-7749ec7548f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:07.564041 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d3975-7ef1-49d2-bfd7-7749ec7548f8 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:07.564331 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d3975-7ef1-49d2-bfd7-7749ec7548f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:07.564613 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d3975-7ef1-49d2-bfd7-7749ec7548f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:07.567352 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d3975-7ef1-49d2-bfd7-7749ec7548f8 None admin] RBAC: Adding query filter params (role.id=a8e4174e3572497f9284c709a119dcff, user.id=a59e517545d741f1b8438c603c29fce2, scope.system=all) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:07.567352 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d3975-7ef1-49d2-bfd7-7749ec7548f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:07.567352 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d3975-7ef1-49d2-bfd7-7749ec7548f8 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:07.579637 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 137/282] 10.222.0.41 () {56 vars in 1308 bytes} [Fri Jun 3 16:02:07 2022] GET /identity/v3/role_assignments?user.id=a59e517545d741f1b8438c603c29fce2&scope.system=all&role.id=a8e4174e3572497f9284c709a119dcff => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:07.635168 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de48f38-f4b2-443f-9138-c6c5c6a08d3e None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:07.637765 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:07.674572 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc8a5fa-e443-4787-99ee-3d89d151d426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:07.675161 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc8a5fa-e443-4787-99ee-3d89d151d426 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:07.675483 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc8a5fa-e443-4787-99ee-3d89d151d426 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:07.675762 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc8a5fa-e443-4787-99ee-3d89d151d426 None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:07.676522 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc8a5fa-e443-4787-99ee-3d89d151d426 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:07.676831 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc8a5fa-e443-4787-99ee-3d89d151d426 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:07.678836 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc8a5fa-e443-4787-99ee-3d89d151d426 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:07.688323 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 146/283] 10.222.0.41 () {56 vars in 1101 bytes} [Fri Jun 3 16:02:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:02:07.694421 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d3975-7ef1-49d2-bfd7-7749ec7548f8 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:07.840495 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d3975-7ef1-49d2-bfd7-7749ec7548f8 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:07.879085 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca04991-3fc4-4c37-b249-1f05cd223c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:07.880460 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca04991-3fc4-4c37-b249-1f05cd223c5d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:07.880942 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca04991-3fc4-4c37-b249-1f05cd223c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:07.881408 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca04991-3fc4-4c37-b249-1f05cd223c5d None admin] PATH_INFO: `/v3/users/22050a6f911c4f699488be5e1bcae98e` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:07.882932 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca04991-3fc4-4c37-b249-1f05cd223c5d None admin] RBAC: Authorizing `identity:get_user(user_id=22050a6f911c4f699488be5e1bcae98e)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:07.884619 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca04991-3fc4-4c37-b249-1f05cd223c5d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:07.886586 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 138/284] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:02:07 2022] GET /identity/v3/users/22050a6f911c4f699488be5e1bcae98e => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:07.892155 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc8a5fa-e443-4787-99ee-3d89d151d426 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:08.059811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc8a5fa-e443-4787-99ee-3d89d151d426 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:08.106245 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94837ede-0c05-46f2-9c97-701648981887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:08.107099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94837ede-0c05-46f2-9c97-701648981887 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:08.107406 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94837ede-0c05-46f2-9c97-701648981887 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:08.107679 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94837ede-0c05-46f2-9c97-701648981887 None admin] PATH_INFO: `/v3/projects/23d03cc631264a3faa207c1b3bded8e3` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:08.112025 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94837ede-0c05-46f2-9c97-701648981887 None admin] RBAC: Authorizing `identity:get_project(project_id=23d03cc631264a3faa207c1b3bded8e3)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:08.113465 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94837ede-0c05-46f2-9c97-701648981887 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:08.115213 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 147/285] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:02:07 2022] GET /identity/v3/projects/23d03cc631264a3faa207c1b3bded8e3 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:08.120901 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca04991-3fc4-4c37-b249-1f05cd223c5d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:08.211210 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:08.292753 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca04991-3fc4-4c37-b249-1f05cd223c5d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:08.344075 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f55891-7002-4664-a36b-fcbf4456103d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:08.345303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f55891-7002-4664-a36b-fcbf4456103d None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:08.345841 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f55891-7002-4664-a36b-fcbf4456103d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:08.346308 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f55891-7002-4664-a36b-fcbf4456103d None admin] PATH_INFO: `/v3/projects/23d03cc631264a3faa207c1b3bded8e3/users/22050a6f911c4f699488be5e1bcae98e/roles/23e883fa91ba4c4f8fa46a1414e9761a` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:08.348531 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f55891-7002-4664-a36b-fcbf4456103d None admin] RBAC: Authorizing `identity:create_grant(project_id=23d03cc631264a3faa207c1b3bded8e3, user_id=22050a6f911c4f699488be5e1bcae98e, role_id=23e883fa91ba4c4f8fa46a1414e9761a)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:08.350213 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f55891-7002-4664-a36b-fcbf4456103d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:08.388756 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 139/286] 10.222.0.41 () {58 vars in 1439 bytes} [Fri Jun 3 16:02:08 2022] PUT /identity/v3/projects/23d03cc631264a3faa207c1b3bded8e3/users/22050a6f911c4f699488be5e1bcae98e/roles/23e883fa91ba4c4f8fa46a1414e9761a => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:02:08.632082 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ac7d4-2403-454c-b438-9d4b55096ae7 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:08.632730 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ac7d4-2403-454c-b438-9d4b55096ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:08.633212 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ac7d4-2403-454c-b438-9d4b55096ae7 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:08.634762 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 148/287] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:08.640456 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c6664-fb38-4b1a-8255-03e84a789d9a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:08.641050 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c6664-fb38-4b1a-8255-03e84a789d9a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:08.641523 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c6664-fb38-4b1a-8255-03e84a789d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:08.660391 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-296c6664-fb38-4b1a-8255-03e84a789d9a None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:08.662257 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-296c6664-fb38-4b1a-8255-03e84a789d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:08.749630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 140/288] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:08.773469 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312ac7d4-2403-454c-b438-9d4b55096ae7 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:08.812036 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:08.892683 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312ac7d4-2403-454c-b438-9d4b55096ae7 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:08.940979 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e379fa83-8d3f-4928-960a-4d97ae59d0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:08.941781 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379fa83-8d3f-4928-960a-4d97ae59d0b0 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:08.942062 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379fa83-8d3f-4928-960a-4d97ae59d0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:08.942339 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379fa83-8d3f-4928-960a-4d97ae59d0b0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:08.957583 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e379fa83-8d3f-4928-960a-4d97ae59d0b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:08.965600 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-e379fa83-8d3f-4928-960a-4d97ae59d0b0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 16:02:08.965600 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:08.965600 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:08.965600 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:08.965600 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:08.965600 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:08.965600 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:08.965600 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:08.965600 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:08.965600 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:08.965600 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:08.965600 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:08.965600 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:08.965600 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:08.965600 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:08.965600 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:08.966546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:08.966546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:08.966546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:08.966546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:08.966546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:08.966546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:08.966546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:08.966546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:08.966546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:08.966546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:08.966546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:08.966546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:08.966546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:08.966546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:08.966546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:08.966546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:08.967371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:08.967371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:08.967371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:08.967371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:08.967371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:08.967371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:08.967371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:08.967371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:08.967371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:08.967371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:08.967371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 16:02:08.967371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:08.967371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 149/289] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:02:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:08.975019 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c6664-fb38-4b1a-8255-03e84a789d9a None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:09.033183 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:09.101938 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c6664-fb38-4b1a-8255-03e84a789d9a None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:09.142899 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd072c6b-e150-440e-966e-dc2816b279b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:09.143517 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd072c6b-e150-440e-966e-dc2816b279b3 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:09.143804 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd072c6b-e150-440e-966e-dc2816b279b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:09.144084 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd072c6b-e150-440e-966e-dc2816b279b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:09.144839 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd072c6b-e150-440e-966e-dc2816b279b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:09.145320 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd072c6b-e150-440e-966e-dc2816b279b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:09.146746 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd072c6b-e150-440e-966e-dc2816b279b3 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:09.155806 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 141/290] 10.222.0.41 () {56 vars in 1101 bytes} [Fri Jun 3 16:02:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:09.161912 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e379fa83-8d3f-4928-960a-4d97ae59d0b0 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:09.275942 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:09.278854 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e379fa83-8d3f-4928-960a-4d97ae59d0b0 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:09.315426 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d55ca2e-4876-43db-9db0-88c5b37a6433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:09.316366 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d55ca2e-4876-43db-9db0-88c5b37a6433 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:09.316651 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d55ca2e-4876-43db-9db0-88c5b37a6433 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:09.316930 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d55ca2e-4876-43db-9db0-88c5b37a6433 None admin] PATH_INFO: `/v3/users/1f833b9b107746d89d7019d9034a5843` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:09.318254 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d55ca2e-4876-43db-9db0-88c5b37a6433 None admin] RBAC: Authorizing `identity:get_user(user_id=1f833b9b107746d89d7019d9034a5843)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:09.319832 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d55ca2e-4876-43db-9db0-88c5b37a6433 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:09.321875 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 150/291] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:02:09 2022] GET /identity/v3/users/1f833b9b107746d89d7019d9034a5843 => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:09.332666 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd072c6b-e150-440e-966e-dc2816b279b3 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:09.456951 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:09.465204 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd072c6b-e150-440e-966e-dc2816b279b3 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:09.522528 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5899d4-0b9c-4df0-ad99-a50b5d65dcb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:09.523185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5899d4-0b9c-4df0-ad99-a50b5d65dcb8 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:09.523491 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5899d4-0b9c-4df0-ad99-a50b5d65dcb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:09.523768 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5899d4-0b9c-4df0-ad99-a50b5d65dcb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:09.651475 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5899d4-0b9c-4df0-ad99-a50b5d65dcb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:09.652978 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5899d4-0b9c-4df0-ad99-a50b5d65dcb8 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:09.792611 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:09.798446 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 142/292] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:02:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 16:02:09.804200 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d55ca2e-4876-43db-9db0-88c5b37a6433 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:10.007148 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9e2ee-cd73-4557-85be-1de87a8db182 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:10.007723 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9e2ee-cd73-4557-85be-1de87a8db182 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:10.008118 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9e2ee-cd73-4557-85be-1de87a8db182 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:10.009807 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 143/293] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:10.015476 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba82569-0b0f-4cce-91d0-54889b2212c1 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:10.016039 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba82569-0b0f-4cce-91d0-54889b2212c1 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:10.016538 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba82569-0b0f-4cce-91d0-54889b2212c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:10.036332 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-3ba82569-0b0f-4cce-91d0-54889b2212c1 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:10.037777 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3ba82569-0b0f-4cce-91d0-54889b2212c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:10.041169 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d55ca2e-4876-43db-9db0-88c5b37a6433 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:10.077851 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80713c6f-2fb0-4911-936d-92672cb19b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:10.078628 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80713c6f-2fb0-4911-936d-92672cb19b39 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:10.078914 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80713c6f-2fb0-4911-936d-92672cb19b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:10.079186 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80713c6f-2fb0-4911-936d-92672cb19b39 None admin] PATH_INFO: `/v3/projects/23d03cc631264a3faa207c1b3bded8e3` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:10.080403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80713c6f-2fb0-4911-936d-92672cb19b39 None admin] RBAC: Authorizing `identity:get_project(project_id=23d03cc631264a3faa207c1b3bded8e3)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:10.081940 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80713c6f-2fb0-4911-936d-92672cb19b39 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:10.084067 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 151/294] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:02:09 2022] GET /identity/v3/projects/23d03cc631264a3faa207c1b3bded8e3 => generated 302 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:10.085256 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cda46-c64a-413c-9988-f3506e9a04e4 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:10.085542 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cda46-c64a-413c-9988-f3506e9a04e4 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:10.085817 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cda46-c64a-413c-9988-f3506e9a04e4 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:10.087026 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 152/295] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:10.091898 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65cda46-c64a-413c-9988-f3506e9a04e4 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:10.116145 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 144/296] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 16:02:10.125264 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a2089-dc66-4401-a9ed-ee5492f57013 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:10.125693 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a2089-dc66-4401-a9ed-ee5492f57013 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:10.126077 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a2089-dc66-4401-a9ed-ee5492f57013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:10.127760 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:10.160357 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-905a2089-dc66-4401-a9ed-ee5492f57013 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:10.161609 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-905a2089-dc66-4401-a9ed-ee5492f57013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:10.209954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65cda46-c64a-413c-9988-f3506e9a04e4 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:10.256772 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbfa728-4ab3-464a-9858-a2710fefcc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:10.257658 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfa728-4ab3-464a-9858-a2710fefcc44 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:10.258149 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfa728-4ab3-464a-9858-a2710fefcc44 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:10.258304 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 145/297] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:10.258836 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfa728-4ab3-464a-9858-a2710fefcc44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:10.260318 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbfa728-4ab3-464a-9858-a2710fefcc44 None admin] RBAC: Adding query filter params (role.id=a8e4174e3572497f9284c709a119dcff, scope.project.id=23d03cc631264a3faa207c1b3bded8e3, user.id=1f833b9b107746d89d7019d9034a5843) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:10.263112 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbfa728-4ab3-464a-9858-a2710fefcc44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:10.263942 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a2089-dc66-4401-a9ed-ee5492f57013 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:10.265483 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbfa728-4ab3-464a-9858-a2710fefcc44 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:10.272368 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 153/298] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:02:10 2022] GET /identity/v3/role_assignments?user.id=1f833b9b107746d89d7019d9034a5843&scope.project.id=23d03cc631264a3faa207c1b3bded8e3&role.id=a8e4174e3572497f9284c709a119dcff => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:10.294095 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbfa728-4ab3-464a-9858-a2710fefcc44 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:10.338170 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:10.385892 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a2089-dc66-4401-a9ed-ee5492f57013 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:10.426526 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6574f8df-1a55-4172-8cc8-2b122ab5465d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:10.428272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574f8df-1a55-4172-8cc8-2b122ab5465d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:10.428733 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574f8df-1a55-4172-8cc8-2b122ab5465d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:10.429180 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574f8df-1a55-4172-8cc8-2b122ab5465d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:10.437498 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:10.451549 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbfa728-4ab3-464a-9858-a2710fefcc44 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:10.455667 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6574f8df-1a55-4172-8cc8-2b122ab5465d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:10.472226 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-6574f8df-1a55-4172-8cc8-2b122ab5465d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 16:02:10.472226 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:10.472226 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:10.472226 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:10.472226 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:10.472226 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:10.472226 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:10.472226 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:10.472226 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:10.472226 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:10.472226 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:10.472226 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:10.472226 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:10.472226 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:10.472226 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:10.472226 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:10.474145 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:10.474145 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:10.474145 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:10.474145 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:10.474145 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:10.474145 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:10.474145 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:10.474145 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:10.474145 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:10.474145 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:10.474145 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:10.474145 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:10.474145 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:10.474145 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:10.474145 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:10.474145 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:10.474942 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:10.474942 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:10.474942 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:10.474942 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:10.474942 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:10.474942 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:10.474942 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:10.474942 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:10.474942 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:10.474942 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:10.474942 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 16:02:10.474942 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:10.474942 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 146/299] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:02:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:10.479992 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6574f8df-1a55-4172-8cc8-2b122ab5465d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:10.495790 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d5c858-585a-4bd8-84d5-c2e0d1dae4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:10.496981 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5c858-585a-4bd8-84d5-c2e0d1dae4d4 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:10.497468 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5c858-585a-4bd8-84d5-c2e0d1dae4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:10.497967 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5c858-585a-4bd8-84d5-c2e0d1dae4d4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:10.499440 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d5c858-585a-4bd8-84d5-c2e0d1dae4d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:10.501109 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d5c858-585a-4bd8-84d5-c2e0d1dae4d4 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:10.503912 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 154/300] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:02:10.510658 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d5c858-585a-4bd8-84d5-c2e0d1dae4d4 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:10.625470 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d5c858-585a-4bd8-84d5-c2e0d1dae4d4 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:10.633839 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6574f8df-1a55-4172-8cc8-2b122ab5465d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:10.651495 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:10.664613 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeaa739-293a-4426-b570-0598eb9e5f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:10.665423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeaa739-293a-4426-b570-0598eb9e5f01 None admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:10.665712 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeaa739-293a-4426-b570-0598eb9e5f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:10.666040 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeaa739-293a-4426-b570-0598eb9e5f01 None admin] PATH_INFO: `/v3/groups` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:10.666863 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeaa739-293a-4426-b570-0598eb9e5f01 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:10.668326 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeaa739-293a-4426-b570-0598eb9e5f01 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:10.671454 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f271201-29d1-4920-875b-99107b4e8ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:10.672133 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f271201-29d1-4920-875b-99107b4e8ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:10.672424 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f271201-29d1-4920-875b-99107b4e8ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:10.672705 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f271201-29d1-4920-875b-99107b4e8ff8 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:10.673586 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f271201-29d1-4920-875b-99107b4e8ff8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:10.673898 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f271201-29d1-4920-875b-99107b4e8ff8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:10.675591 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f271201-29d1-4920-875b-99107b4e8ff8 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:10.685381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 155/301] 10.222.0.41 () {60 vars in 1134 bytes} [Fri Jun 3 16:02:10 2022] POST /identity/v3/groups => generated 229 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:02:10.686572 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 147/302] 10.222.0.41 () {56 vars in 1101 bytes} [Fri Jun 3 16:02:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:02:10.686763 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989b1b8-bb50-405f-93f3-ad331b4233a0 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:10.687052 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989b1b8-bb50-405f-93f3-ad331b4233a0 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:10.687408 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989b1b8-bb50-405f-93f3-ad331b4233a0 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:10.688911 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 156/303] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:10.693513 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f271201-29d1-4920-875b-99107b4e8ff8 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:10.694882 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e657878-cdc0-4d62-b33c-ef9fa5ec709b None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:10.695185 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e657878-cdc0-4d62-b33c-ef9fa5ec709b None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:10.695499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e657878-cdc0-4d62-b33c-ef9fa5ec709b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:10.718530 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-0e657878-cdc0-4d62-b33c-ef9fa5ec709b None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:10.719975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-0e657878-cdc0-4d62-b33c-ef9fa5ec709b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:10.741614 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:10.804000 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 157/304] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 16:02:10.828263 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e657878-cdc0-4d62-b33c-ef9fa5ec709b None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:10.828866 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f271201-29d1-4920-875b-99107b4e8ff8 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:10.878025 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5f393-4475-4e8a-8c17-c72cf1e97604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:10.879480 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5f393-4475-4e8a-8c17-c72cf1e97604 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:10.879839 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5f393-4475-4e8a-8c17-c72cf1e97604 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:10.880137 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5f393-4475-4e8a-8c17-c72cf1e97604 None admin] PATH_INFO: `/v3/users/d652bf1b187f4d489009eceeee5c12ab` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:10.897480 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5f393-4475-4e8a-8c17-c72cf1e97604 None admin] RBAC: Authorizing `identity:get_user(user_id=d652bf1b187f4d489009eceeee5c12ab)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:10.899167 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5f393-4475-4e8a-8c17-c72cf1e97604 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:10.900960 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 148/305] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:02:10 2022] GET /identity/v3/users/d652bf1b187f4d489009eceeee5c12ab => generated 293 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:10.907604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5f393-4475-4e8a-8c17-c72cf1e97604 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:10.954623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e657878-cdc0-4d62-b33c-ef9fa5ec709b None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:10.962339 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:11.005560 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c9236b-3164-44df-a503-ecf401593591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:11.005772 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9236b-3164-44df-a503-ecf401593591 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:11.006208 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9236b-3164-44df-a503-ecf401593591 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:11.006473 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9236b-3164-44df-a503-ecf401593591 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:11.019383 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c9236b-3164-44df-a503-ecf401593591 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:11.027748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-42c9236b-3164-44df-a503-ecf401593591 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 16:02:11.027748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:11.027748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:11.027748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:11.027748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:11.027748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:11.027748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:11.027748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:11.027748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:11.027748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:11.027748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:11.027748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:11.027748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:11.027748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:11.027748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:11.027748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:11.028639 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:11.028639 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:11.028639 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:11.028639 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:11.028639 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:11.028639 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:11.028639 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:11.028639 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:11.028639 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:11.028639 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:11.028639 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:11.028639 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:11.028639 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:11.028639 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:11.028639 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:11.028639 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:11.030112 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:11.030112 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:11.030112 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:11.030112 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:11.030112 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:11.030112 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:11.030112 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:11.030112 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:11.030112 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:11.030112 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:11.030112 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 16:02:11.030112 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:11.030112 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 158/306] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:02:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:02:11.034959 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c9236b-3164-44df-a503-ecf401593591 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:11.049354 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5f393-4475-4e8a-8c17-c72cf1e97604 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:11.062584 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:11.093877 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1721b-5571-4820-bc5f-21bc7f67677e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:11.094942 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1721b-5571-4820-bc5f-21bc7f67677e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:11.095343 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1721b-5571-4820-bc5f-21bc7f67677e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:11.095876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1721b-5571-4820-bc5f-21bc7f67677e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:11.097133 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c1721b-5571-4820-bc5f-21bc7f67677e None admin] RBAC: Adding query filter params (role.id=23e883fa91ba4c4f8fa46a1414e9761a, user.id=d652bf1b187f4d489009eceeee5c12ab, scope.system=all) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:11.097553 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c1721b-5571-4820-bc5f-21bc7f67677e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:11.100145 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c1721b-5571-4820-bc5f-21bc7f67677e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:11.107114 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 149/307] 10.222.0.41 () {56 vars in 1308 bytes} [Fri Jun 3 16:02:10 2022] GET /identity/v3/role_assignments?user.id=d652bf1b187f4d489009eceeee5c12ab&scope.system=all&role.id=23e883fa91ba4c4f8fa46a1414e9761a => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:02:11.158949 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c9236b-3164-44df-a503-ecf401593591 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:11.197194 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe4981-c69b-40d6-bd00-bc7d6d5eceae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:11.197884 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe4981-c69b-40d6-bd00-bc7d6d5eceae None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:11.198170 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe4981-c69b-40d6-bd00-bc7d6d5eceae None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:11.198448 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe4981-c69b-40d6-bd00-bc7d6d5eceae None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:11.199215 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe4981-c69b-40d6-bd00-bc7d6d5eceae None admin] RBAC: Adding query filter params (name=member) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:11.199559 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe4981-c69b-40d6-bd00-bc7d6d5eceae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:11.200966 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe4981-c69b-40d6-bd00-bc7d6d5eceae None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:11.215783 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 159/308] 10.222.0.41 () {56 vars in 1101 bytes} [Fri Jun 3 16:02:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:02:11.224523 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1721b-5571-4820-bc5f-21bc7f67677e None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:11.326308 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe71903-d7d4-47ce-aec2-d3d23d9cfd6d None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:11.326620 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe71903-d7d4-47ce-aec2-d3d23d9cfd6d None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:11.326877 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe71903-d7d4-47ce-aec2-d3d23d9cfd6d None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:11.328483 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 160/309] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:11.334010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb8b0a8-3589-4706-aa2b-554a226284c1 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:11.334307 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb8b0a8-3589-4706-aa2b-554a226284c1 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:11.334577 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb8b0a8-3589-4706-aa2b-554a226284c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:11.344247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1721b-5571-4820-bc5f-21bc7f67677e None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:11.345809 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:11.353696 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-5bb8b0a8-3589-4706-aa2b-554a226284c1 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:11.355008 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-5bb8b0a8-3589-4706-aa2b-554a226284c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:11.384909 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b673d0-5d9d-4d3d-975c-7498172cf0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:11.385789 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b673d0-5d9d-4d3d-975c-7498172cf0f4 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:11.386072 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b673d0-5d9d-4d3d-975c-7498172cf0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:11.386349 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b673d0-5d9d-4d3d-975c-7498172cf0f4 None admin] PATH_INFO: `/v3/users/a59e517545d741f1b8438c603c29fce2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:11.387589 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b673d0-5d9d-4d3d-975c-7498172cf0f4 None admin] RBAC: Authorizing `identity:get_user(user_id=a59e517545d741f1b8438c603c29fce2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:11.389042 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b673d0-5d9d-4d3d-975c-7498172cf0f4 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:11.395387 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 150/310] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:02:11 2022] GET /identity/v3/users/a59e517545d741f1b8438c603c29fce2 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:11.396719 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b673d0-5d9d-4d3d-975c-7498172cf0f4 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:11.432983 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 161/311] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:11.455840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb8b0a8-3589-4706-aa2b-554a226284c1 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:11.459412 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:11.521151 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b673d0-5d9d-4d3d-975c-7498172cf0f4 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:11.559943 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e53ea-370c-4c47-b776-36599e928730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:11.561196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e53ea-370c-4c47-b776-36599e928730 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:11.561727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e53ea-370c-4c47-b776-36599e928730 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:11.562328 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e53ea-370c-4c47-b776-36599e928730 None admin] PATH_INFO: `/v3/system/users/a59e517545d741f1b8438c603c29fce2/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:11.564068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39e53ea-370c-4c47-b776-36599e928730 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a59e517545d741f1b8438c603c29fce2, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:11.565858 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39e53ea-370c-4c47-b776-36599e928730 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:11.576484 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 151/312] 10.222.0.41 () {58 vars in 1334 bytes} [Fri Jun 3 16:02:11 2022] PUT /identity/v3/system/users/a59e517545d741f1b8438c603c29fce2/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:02:11.584391 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb8b0a8-3589-4706-aa2b-554a226284c1 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:11.626549 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440e5b3-1eb9-49e8-ac75-bfb6098d3ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:11.627453 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440e5b3-1eb9-49e8-ac75-bfb6098d3ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:11.627736 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440e5b3-1eb9-49e8-ac75-bfb6098d3ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:11.628004 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440e5b3-1eb9-49e8-ac75-bfb6098d3ac3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:11.640132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9440e5b3-1eb9-49e8-ac75-bfb6098d3ac3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:11.648672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-9440e5b3-1eb9-49e8-ac75-bfb6098d3ac3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 16:02:11.648672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:11.648672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:11.648672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:11.648672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:11.648672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:11.648672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:11.648672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:11.648672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:11.648672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:11.648672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:11.648672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:11.648672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:11.648672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:11.648672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:11.648672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:11.649615 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:11.649615 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:11.649615 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:11.649615 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:11.649615 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:11.649615 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:11.649615 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:11.649615 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:11.649615 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:11.649615 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:11.649615 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:11.649615 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:11.649615 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:11.649615 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:11.649615 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:11.649615 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:11.650455 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:11.650455 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:11.650455 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:11.650455 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:11.650455 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:11.650455 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:11.650455 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:11.650455 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:11.650455 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:11.650455 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:11.650455 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 16:02:11.650455 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:11.650455 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 162/313] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:02:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:02:11.658183 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e53ea-370c-4c47-b776-36599e928730 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:11.697948 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:11.775052 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e53ea-370c-4c47-b776-36599e928730 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:11.813719 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640eb04b-6b35-43f9-a919-76341752ac0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:11.814738 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640eb04b-6b35-43f9-a919-76341752ac0f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:11.815221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640eb04b-6b35-43f9-a919-76341752ac0f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:11.815716 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640eb04b-6b35-43f9-a919-76341752ac0f None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:11.816503 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640eb04b-6b35-43f9-a919-76341752ac0f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:11.816826 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640eb04b-6b35-43f9-a919-76341752ac0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:11.818259 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640eb04b-6b35-43f9-a919-76341752ac0f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:11.827147 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 152/314] 10.222.0.41 () {56 vars in 1101 bytes} [Fri Jun 3 16:02:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:11.832473 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440e5b3-1eb9-49e8-ac75-bfb6098d3ac3 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:11.897591 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:11.937102 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440e5b3-1eb9-49e8-ac75-bfb6098d3ac3 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:11.972336 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f928f8-e1d6-4936-9341-874bd23b6eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:11.973131 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f928f8-e1d6-4936-9341-874bd23b6eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:11.973415 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f928f8-e1d6-4936-9341-874bd23b6eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:11.973724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f928f8-e1d6-4936-9341-874bd23b6eb0 None admin] PATH_INFO: `/v3/users/22050a6f911c4f699488be5e1bcae98e` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:11.974939 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f928f8-e1d6-4936-9341-874bd23b6eb0 None admin] RBAC: Authorizing `identity:get_user(user_id=22050a6f911c4f699488be5e1bcae98e)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:11.976425 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f928f8-e1d6-4936-9341-874bd23b6eb0 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:11.978055 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 163/315] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:02:11 2022] GET /identity/v3/users/22050a6f911c4f699488be5e1bcae98e => generated 297 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:02:11.983620 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640eb04b-6b35-43f9-a919-76341752ac0f None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:12.091127 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:12.097863 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640eb04b-6b35-43f9-a919-76341752ac0f None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:12.134279 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e2e86-ff12-47bf-93ec-6c4f53c57f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:12.135218 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e2e86-ff12-47bf-93ec-6c4f53c57f4b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:12.135712 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e2e86-ff12-47bf-93ec-6c4f53c57f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:12.136163 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e2e86-ff12-47bf-93ec-6c4f53c57f4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:12.207685 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283e2e86-ff12-47bf-93ec-6c4f53c57f4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:12.209965 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283e2e86-ff12-47bf-93ec-6c4f53c57f4b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:12.314740 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:12.323165 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 153/316] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:02:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 16:02:12.328512 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f928f8-e1d6-4936-9341-874bd23b6eb0 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:12.430808 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f928f8-e1d6-4936-9341-874bd23b6eb0 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:12.461670 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:12.467962 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52952b05-4446-4127-b6e3-8507008da251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:12.468750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52952b05-4446-4127-b6e3-8507008da251 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:12.469359 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52952b05-4446-4127-b6e3-8507008da251 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:12.469640 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52952b05-4446-4127-b6e3-8507008da251 None admin] PATH_INFO: `/v3/projects/23d03cc631264a3faa207c1b3bded8e3` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:12.470833 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52952b05-4446-4127-b6e3-8507008da251 None admin] RBAC: Authorizing `identity:get_project(project_id=23d03cc631264a3faa207c1b3bded8e3)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:12.472330 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52952b05-4446-4127-b6e3-8507008da251 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:12.474072 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 164/317] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:02:12 2022] GET /identity/v3/projects/23d03cc631264a3faa207c1b3bded8e3 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:12.480065 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e2e86-ff12-47bf-93ec-6c4f53c57f4b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:12.586343 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e2e86-ff12-47bf-93ec-6c4f53c57f4b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:12.622412 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b50dd0-2ee1-4dc0-b17d-711339c6ddb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:12.623070 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b50dd0-2ee1-4dc0-b17d-711339c6ddb8 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:12.623429 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b50dd0-2ee1-4dc0-b17d-711339c6ddb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:12.623713 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b50dd0-2ee1-4dc0-b17d-711339c6ddb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:12.624618 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b50dd0-2ee1-4dc0-b17d-711339c6ddb8 None admin] RBAC: Adding query filter params (role.id=23e883fa91ba4c4f8fa46a1414e9761a, scope.project.id=23d03cc631264a3faa207c1b3bded8e3, user.id=22050a6f911c4f699488be5e1bcae98e) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:12.624937 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b50dd0-2ee1-4dc0-b17d-711339c6ddb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:12.626395 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b50dd0-2ee1-4dc0-b17d-711339c6ddb8 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:12.633376 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 154/318] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:02:12 2022] GET /identity/v3/role_assignments?user.id=22050a6f911c4f699488be5e1bcae98e&scope.project.id=23d03cc631264a3faa207c1b3bded8e3&role.id=23e883fa91ba4c4f8fa46a1414e9761a => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:13.151397 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7aba3-6724-45ba-a540-cacb2f034a6c None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:13.151940 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7aba3-6724-45ba-a540-cacb2f034a6c None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:13.151940 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7aba3-6724-45ba-a540-cacb2f034a6c None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:13.153335 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 165/319] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:13.158543 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a624b-5f89-4d76-9c6d-b6aa3e2f6de3 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:13.158836 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a624b-5f89-4d76-9c6d-b6aa3e2f6de3 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:13.159111 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a624b-5f89-4d76-9c6d-b6aa3e2f6de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:13.175383 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-781a624b-5f89-4d76-9c6d-b6aa3e2f6de3 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:13.176549 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-781a624b-5f89-4d76-9c6d-b6aa3e2f6de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:13.234994 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:13.255419 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 155/320] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:13.269954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b7aba3-6724-45ba-a540-cacb2f034a6c None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:13.369120 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b7aba3-6724-45ba-a540-cacb2f034a6c None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:13.402071 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca6480-ec52-4e00-87f6-9d2637fec5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:13.402789 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca6480-ec52-4e00-87f6-9d2637fec5ad None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:13.403071 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca6480-ec52-4e00-87f6-9d2637fec5ad None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:13.403374 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca6480-ec52-4e00-87f6-9d2637fec5ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:13.404787 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ca6480-ec52-4e00-87f6-9d2637fec5ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:13.406200 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ca6480-ec52-4e00-87f6-9d2637fec5ad None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:13.408039 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 166/321] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:13.413395 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a624b-5f89-4d76-9c6d-b6aa3e2f6de3 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:13.438262 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378323f7-a630-4624-b94d-eea5162746ff None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:13.438556 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378323f7-a630-4624-b94d-eea5162746ff None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:13.438839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378323f7-a630-4624-b94d-eea5162746ff None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:13.440299 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 167/322] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 16:02:13.445760 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b44a0a-8180-4df7-a069-aeada3503872 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:13.446044 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b44a0a-8180-4df7-a069-aeada3503872 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:13.446320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b44a0a-8180-4df7-a069-aeada3503872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:13.463699 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-e0b44a0a-8180-4df7-a069-aeada3503872 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:13.464896 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-e0b44a0a-8180-4df7-a069-aeada3503872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:13.527000 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a624b-5f89-4d76-9c6d-b6aa3e2f6de3 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:13.533071 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:13.540087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 168/323] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 16:02:13.562042 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b44a0a-8180-4df7-a069-aeada3503872 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:13.562971 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad041347-9894-463f-83a3-e0b0a6599bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:13.563667 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad041347-9894-463f-83a3-e0b0a6599bfa None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:13.563955 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad041347-9894-463f-83a3-e0b0a6599bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:13.564257 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad041347-9894-463f-83a3-e0b0a6599bfa None admin] PATH_INFO: `/v3/groups` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:13.564916 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad041347-9894-463f-83a3-e0b0a6599bfa None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:13.566358 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad041347-9894-463f-83a3-e0b0a6599bfa None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:13.584071 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 156/324] 10.222.0.41 () {60 vars in 1134 bytes} [Fri Jun 3 16:02:13 2022] POST /identity/v3/groups => generated 226 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:02:13.679014 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b44a0a-8180-4df7-a069-aeada3503872 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:13.717954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ceb4fc-d39d-4703-b2e4-a17ccfb072bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:13.718771 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ceb4fc-d39d-4703-b2e4-a17ccfb072bf None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:13.719055 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ceb4fc-d39d-4703-b2e4-a17ccfb072bf None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:13.719352 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ceb4fc-d39d-4703-b2e4-a17ccfb072bf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:13.730653 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ceb4fc-d39d-4703-b2e4-a17ccfb072bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:13.738479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-b1ceb4fc-d39d-4703-b2e4-a17ccfb072bf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 16:02:13.738479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:13.738479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:13.738479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:13.738479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:13.738479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:13.738479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:13.738479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:13.738479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:13.738479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:13.738479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:13.738479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:13.738479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:13.738479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:13.738479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:13.738479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:13.739971 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:13.739971 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:13.739971 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:13.739971 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:13.739971 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:13.739971 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:13.739971 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:13.739971 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:13.739971 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:13.739971 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:13.739971 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:13.739971 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:13.739971 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:13.739971 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:13.739971 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:13.739971 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:13.745708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:13.745708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:13.745708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:13.745708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:13.745708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:13.745708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:13.745708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:13.745708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:13.745708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:13.745708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:13.745708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 16:02:13.745708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:13.745708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 169/325] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:02:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:02:13.746351 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad041347-9894-463f-83a3-e0b0a6599bfa None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:13.802426 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:13.858201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad041347-9894-463f-83a3-e0b0a6599bfa None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:13.895500 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15996b2-9a33-4248-95b0-dcd97c0c045b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:13.896418 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15996b2-9a33-4248-95b0-dcd97c0c045b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:13.896709 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15996b2-9a33-4248-95b0-dcd97c0c045b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:13.896991 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15996b2-9a33-4248-95b0-dcd97c0c045b None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:13.897757 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15996b2-9a33-4248-95b0-dcd97c0c045b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:13.898068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15996b2-9a33-4248-95b0-dcd97c0c045b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:13.899480 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15996b2-9a33-4248-95b0-dcd97c0c045b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:13.908524 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 157/326] 10.222.0.41 () {56 vars in 1101 bytes} [Fri Jun 3 16:02:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:13.914308 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ceb4fc-d39d-4703-b2e4-a17ccfb072bf None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:13.993315 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:14.016553 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17eb49-3103-4b18-9604-280ca2f60926 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:14.016906 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17eb49-3103-4b18-9604-280ca2f60926 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:14.017451 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17eb49-3103-4b18-9604-280ca2f60926 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:14.018504 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 158/327] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 16:02:14.023749 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461dda5-d2c6-4c0a-bd7b-d51a16c4a115 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:14.024087 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461dda5-d2c6-4c0a-bd7b-d51a16c4a115 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:14.024315 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461dda5-d2c6-4c0a-bd7b-d51a16c4a115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:14.042992 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-a461dda5-d2c6-4c0a-bd7b-d51a16c4a115 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:14.044216 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a461dda5-d2c6-4c0a-bd7b-d51a16c4a115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:14.044399 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ceb4fc-d39d-4703-b2e4-a17ccfb072bf None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:14.093231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae03a91e-9292-488e-8d3f-f3094675c8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:14.094106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae03a91e-9292-488e-8d3f-f3094675c8bb None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:14.094389 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae03a91e-9292-488e-8d3f-f3094675c8bb None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:14.094662 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae03a91e-9292-488e-8d3f-f3094675c8bb None admin] PATH_INFO: `/v3/users/d652bf1b187f4d489009eceeee5c12ab` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:14.096025 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae03a91e-9292-488e-8d3f-f3094675c8bb None admin] RBAC: Authorizing `identity:get_user(user_id=d652bf1b187f4d489009eceeee5c12ab)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:14.097484 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae03a91e-9292-488e-8d3f-f3094675c8bb None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:14.101023 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 170/328] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:02:13 2022] GET /identity/v3/users/d652bf1b187f4d489009eceeee5c12ab => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:14.105599 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae03a91e-9292-488e-8d3f-f3094675c8bb None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:14.132701 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 159/329] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 16:02:14.156449 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a461dda5-d2c6-4c0a-bd7b-d51a16c4a115 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:14.221438 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:14.251755 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae03a91e-9292-488e-8d3f-f3094675c8bb None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:14.279945 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:14.289517 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a354d7b-17cc-4f4f-b2f2-129fb96f6d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:14.291005 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a354d7b-17cc-4f4f-b2f2-129fb96f6d51 None admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:14.291848 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a354d7b-17cc-4f4f-b2f2-129fb96f6d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:14.292306 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a354d7b-17cc-4f4f-b2f2-129fb96f6d51 None admin] PATH_INFO: `/v3/system/users/d652bf1b187f4d489009eceeee5c12ab/roles/23e883fa91ba4c4f8fa46a1414e9761a` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:14.294077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a354d7b-17cc-4f4f-b2f2-129fb96f6d51 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d652bf1b187f4d489009eceeee5c12ab, role_id=23e883fa91ba4c4f8fa46a1414e9761a)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:14.295699 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a354d7b-17cc-4f4f-b2f2-129fb96f6d51 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:14.296615 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a461dda5-d2c6-4c0a-bd7b-d51a16c4a115 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:14.306995 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 171/330] 10.222.0.41 () {58 vars in 1334 bytes} [Fri Jun 3 16:02:14 2022] PUT /identity/v3/system/users/d652bf1b187f4d489009eceeee5c12ab/roles/23e883fa91ba4c4f8fa46a1414e9761a => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:02:14.342634 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45246f7-4745-48c1-aedc-e6e5f0058095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:14.343495 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45246f7-4745-48c1-aedc-e6e5f0058095 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:14.343783 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45246f7-4745-48c1-aedc-e6e5f0058095 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:14.344111 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45246f7-4745-48c1-aedc-e6e5f0058095 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:14.356904 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45246f7-4745-48c1-aedc-e6e5f0058095 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:14.364910 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-a45246f7-4745-48c1-aedc-e6e5f0058095 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 16:02:14.364910 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:14.364910 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:14.364910 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:14.364910 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:14.364910 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:14.364910 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:14.364910 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:14.364910 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:14.364910 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:14.364910 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:14.364910 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:14.364910 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:14.364910 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:14.364910 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:14.364910 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:14.365850 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:14.365850 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:14.365850 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:14.365850 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:14.365850 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:14.365850 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:14.365850 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:14.365850 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:14.365850 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:14.365850 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:14.365850 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:14.365850 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:14.365850 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:14.365850 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:14.365850 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:14.365850 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:14.366616 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:14.366616 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:14.366616 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:14.366616 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:14.366616 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:14.366616 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:14.366616 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:14.366616 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:14.366616 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:14.366616 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:14.366616 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 16:02:14.366616 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:14.366616 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 160/331] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:02:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:02:14.372276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a354d7b-17cc-4f4f-b2f2-129fb96f6d51 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:14.497102 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a354d7b-17cc-4f4f-b2f2-129fb96f6d51 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:14.538859 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579d130-781a-4de7-bcd5-0a40447b2f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:14.539502 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579d130-781a-4de7-bcd5-0a40447b2f4d None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:14.539789 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579d130-781a-4de7-bcd5-0a40447b2f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:14.540068 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579d130-781a-4de7-bcd5-0a40447b2f4d None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:14.540840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b579d130-781a-4de7-bcd5-0a40447b2f4d None admin] RBAC: Adding query filter params (name=member) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:14.541151 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b579d130-781a-4de7-bcd5-0a40447b2f4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:14.542553 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b579d130-781a-4de7-bcd5-0a40447b2f4d None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:14.554482 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 172/332] 10.222.0.41 () {56 vars in 1101 bytes} [Fri Jun 3 16:02:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:14.560185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45246f7-4745-48c1-aedc-e6e5f0058095 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:14.667151 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45246f7-4745-48c1-aedc-e6e5f0058095 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:14.668709 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:14.703444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf35dc-3e9c-48b3-b706-c510d871d2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:14.704332 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf35dc-3e9c-48b3-b706-c510d871d2f5 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:14.704621 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf35dc-3e9c-48b3-b706-c510d871d2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:14.704898 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf35dc-3e9c-48b3-b706-c510d871d2f5 None admin] PATH_INFO: `/v3/users/a59e517545d741f1b8438c603c29fce2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:14.706115 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbf35dc-3e9c-48b3-b706-c510d871d2f5 None admin] RBAC: Authorizing `identity:get_user(user_id=a59e517545d741f1b8438c603c29fce2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:14.707611 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbf35dc-3e9c-48b3-b706-c510d871d2f5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:14.709591 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 161/333] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:02:14 2022] GET /identity/v3/users/a59e517545d741f1b8438c603c29fce2 => generated 293 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:14.715208 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579d130-781a-4de7-bcd5-0a40447b2f4d None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:14.772120 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:14.824437 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579d130-781a-4de7-bcd5-0a40447b2f4d None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:14.860363 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb8f39-d3d3-4ad6-9c10-4f2235e527ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:14.861022 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb8f39-d3d3-4ad6-9c10-4f2235e527ed None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:14.861314 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb8f39-d3d3-4ad6-9c10-4f2235e527ed None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:14.861593 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb8f39-d3d3-4ad6-9c10-4f2235e527ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:14.862429 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fb8f39-d3d3-4ad6-9c10-4f2235e527ed None admin] RBAC: Adding query filter params (role.id=a8e4174e3572497f9284c709a119dcff, user.id=a59e517545d741f1b8438c603c29fce2, scope.system=all) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:14.862745 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fb8f39-d3d3-4ad6-9c10-4f2235e527ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:14.864833 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fb8f39-d3d3-4ad6-9c10-4f2235e527ed None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:14.876928 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 173/334] 10.222.0.41 () {56 vars in 1308 bytes} [Fri Jun 3 16:02:14 2022] GET /identity/v3/role_assignments?user.id=a59e517545d741f1b8438c603c29fce2&scope.system=all&role.id=a8e4174e3572497f9284c709a119dcff => generated 510 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:16.831060 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af65f13-8ab2-46ea-a8db-c7ae88da7f1d None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:16.832138 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af65f13-8ab2-46ea-a8db-c7ae88da7f1d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:16.832635 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af65f13-8ab2-46ea-a8db-c7ae88da7f1d None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:16.834517 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 162/335] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:16.839435 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c152d-ad50-402c-878a-11f7cf6e5e75 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:16.840289 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c152d-ad50-402c-878a-11f7cf6e5e75 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:16.840784 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c152d-ad50-402c-878a-11f7cf6e5e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:16.864672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-7e4c152d-ad50-402c-878a-11f7cf6e5e75 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:16.866225 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-7e4c152d-ad50-402c-878a-11f7cf6e5e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:16.947721 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 174/336] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:16.953526 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4ce93-0af2-45d9-9640-8b09b52dbbde None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:16.954043 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4ce93-0af2-45d9-9640-8b09b52dbbde None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:16.954524 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4ce93-0af2-45d9-9640-8b09b52dbbde None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:16.956133 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 163/337] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:16.962650 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff7f42-66ee-4dbe-952c-3b538eb199cf None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:16.962945 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff7f42-66ee-4dbe-952c-3b538eb199cf None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:16.963232 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff7f42-66ee-4dbe-952c-3b538eb199cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:16.984237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4ce93-0af2-45d9-9640-8b09b52dbbde None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:16.991098 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-68ff7f42-66ee-4dbe-952c-3b538eb199cf None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:16.991098 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-68ff7f42-66ee-4dbe-952c-3b538eb199cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:17.053726 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:17.109233 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 175/338] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:17.124381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcffae99-fa0c-49b7-aa09-cc0058919089 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:17.124672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcffae99-fa0c-49b7-aa09-cc0058919089 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:17.124946 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcffae99-fa0c-49b7-aa09-cc0058919089 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:17.126537 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 176/339] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:17.131941 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf7cbde-7850-40e5-9acf-ceadbf335eb1 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:17.132231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf7cbde-7850-40e5-9acf-ceadbf335eb1 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:17.132514 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf7cbde-7850-40e5-9acf-ceadbf335eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:17.163531 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-7cf7cbde-7850-40e5-9acf-ceadbf335eb1 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:17.164717 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-7cf7cbde-7850-40e5-9acf-ceadbf335eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:17.207592 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4ce93-0af2-45d9-9640-8b09b52dbbde None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:17.249050 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e52483-7d58-42e1-ab10-c1eeafafc2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:17.249050 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e52483-7d58-42e1-ab10-c1eeafafc2c5 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:17.249050 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e52483-7d58-42e1-ab10-c1eeafafc2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:17.249050 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e52483-7d58-42e1-ab10-c1eeafafc2c5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:17.263318 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e52483-7d58-42e1-ab10-c1eeafafc2c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:17.271284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-81e52483-7d58-42e1-ab10-c1eeafafc2c5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 16:02:17.271284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:17.271284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:17.271284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:17.271284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:17.271284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:17.271284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:17.271284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:17.271284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:17.271284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:17.271284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:17.271284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:17.271284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:17.271284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:17.271284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:17.271284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:17.272223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:17.272223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:17.272223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:17.272223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:17.272223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:17.272223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:17.272223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:17.272223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:17.272223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:17.272223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:17.272223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:17.272223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:17.272223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:17.272223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:17.272223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:17.272223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:17.273604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:17.273604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:17.273604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:17.273604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:17.273604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:17.273604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:17.273604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:17.273604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:17.273604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:17.273604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:17.273604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 16:02:17.273604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:17.273604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 164/340] 10.222.0.41 () {56 vars in 1114 bytes} [Fri Jun 3 16:02:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:17.274320 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e52483-7d58-42e1-ab10-c1eeafafc2c5 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:17.292988 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 177/341] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:17.294299 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf7cbde-7850-40e5-9acf-ceadbf335eb1 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:17.315684 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:17.389200 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e52483-7d58-42e1-ab10-c1eeafafc2c5 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:17.426748 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7201e44f-37db-45ee-9a99-5abf2586ad94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:17.427750 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201e44f-37db-45ee-9a99-5abf2586ad94 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:17.428037 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201e44f-37db-45ee-9a99-5abf2586ad94 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:17.428319 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201e44f-37db-45ee-9a99-5abf2586ad94 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:17.440440 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7201e44f-37db-45ee-9a99-5abf2586ad94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:17.449327 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-7201e44f-37db-45ee-9a99-5abf2586ad94 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 16:02:17.449327 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:17.449327 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:17.449327 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:17.449327 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:17.449327 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:17.449327 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:17.449327 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:17.449327 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:17.449327 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:17.449327 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:17.449327 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:17.449327 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:17.449327 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:17.449327 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:17.449327 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:17.450214 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:17.450214 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:17.450214 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:17.450214 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:17.450214 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:17.450214 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:17.450214 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:17.450214 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:17.450214 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:17.450214 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:17.450214 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:17.450214 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:17.450214 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:17.450214 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:17.450214 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:17.450214 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:17.451001 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:17.451001 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:17.451001 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:17.451001 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:17.451001 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:17.451001 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:17.451001 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:17.451001 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:17.451001 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:17.451001 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:17.451001 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 16:02:17.451001 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:17.451001 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 165/342] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:02:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:17.451695 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7201e44f-37db-45ee-9a99-5abf2586ad94 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:17.559729 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:17.565957 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:17.569657 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf7cbde-7850-40e5-9acf-ceadbf335eb1 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:17.578162 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7201e44f-37db-45ee-9a99-5abf2586ad94 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:17.624788 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7914bc-579d-4126-82fd-dabd7ec1afb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:17.625596 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7914bc-579d-4126-82fd-dabd7ec1afb5 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:17.625880 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7914bc-579d-4126-82fd-dabd7ec1afb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:17.626160 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7914bc-579d-4126-82fd-dabd7ec1afb5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:17.627955 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa077b38-74b1-4934-a79f-ab42208cce64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:17.628628 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa077b38-74b1-4934-a79f-ab42208cce64 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:17.628908 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa077b38-74b1-4934-a79f-ab42208cce64 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:17.629181 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa077b38-74b1-4934-a79f-ab42208cce64 None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:17.631948 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa077b38-74b1-4934-a79f-ab42208cce64 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:17.632269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa077b38-74b1-4934-a79f-ab42208cce64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:17.636949 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa077b38-74b1-4934-a79f-ab42208cce64 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:17.648319 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7914bc-579d-4126-82fd-dabd7ec1afb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:17.657425 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-3d7914bc-579d-4126-82fd-dabd7ec1afb5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 16:02:17.657425 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:17.657425 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:17.657425 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:17.657425 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:17.657425 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:17.657425 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:17.657425 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:17.657425 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:17.657425 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:17.657425 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:17.657425 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:17.657425 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:17.657425 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:17.657425 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:17.657425 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:17.658314 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:17.658314 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:17.658314 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:17.658314 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:17.658314 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:17.658314 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:17.658314 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:17.658314 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:17.658314 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:17.658314 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:17.658314 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:17.658314 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:17.658314 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:17.658314 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:17.658314 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:17.658314 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:17.659567 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:17.659567 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:17.659567 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:17.659567 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:17.659567 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:17.659567 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:17.659567 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:17.659567 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:17.659567 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:17.659567 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:17.659567 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 16:02:17.659567 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:17.659567 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 166/343] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:02:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:17.661262 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 178/344] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:02:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:17.661526 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7914bc-579d-4126-82fd-dabd7ec1afb5 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:17.667324 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa077b38-74b1-4934-a79f-ab42208cce64 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:17.840759 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa077b38-74b1-4934-a79f-ab42208cce64 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:17.873035 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7914bc-579d-4126-82fd-dabd7ec1afb5 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:17.898477 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1547c4-d62d-41a3-a5b0-435cc762174e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:17.899060 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1547c4-d62d-41a3-a5b0-435cc762174e None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:17.899366 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1547c4-d62d-41a3-a5b0-435cc762174e None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:17.899649 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1547c4-d62d-41a3-a5b0-435cc762174e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:17.933688 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee20c63-3171-49bc-8671-e0d66a424f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:17.934467 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee20c63-3171-49bc-8671-e0d66a424f05 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:17.934758 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee20c63-3171-49bc-8671-e0d66a424f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:17.935039 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee20c63-3171-49bc-8671-e0d66a424f05 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:17.935827 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee20c63-3171-49bc-8671-e0d66a424f05 None admin] RBAC: Adding query filter params (name=member) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:17.936142 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee20c63-3171-49bc-8671-e0d66a424f05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:17.937529 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee20c63-3171-49bc-8671-e0d66a424f05 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:17.950727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 167/345] 10.222.0.41 () {56 vars in 1101 bytes} [Fri Jun 3 16:02:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:17.951529 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee20c63-3171-49bc-8671-e0d66a424f05 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:17.961874 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:17.970688 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:18.015598 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1547c4-d62d-41a3-a5b0-435cc762174e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:18.017058 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1547c4-d62d-41a3-a5b0-435cc762174e None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:18.070696 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee20c63-3171-49bc-8671-e0d66a424f05 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:18.110020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dfc4d3-49ac-432d-9f49-f9e369c2defa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:18.111136 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfc4d3-49ac-432d-9f49-f9e369c2defa None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:18.111630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfc4d3-49ac-432d-9f49-f9e369c2defa None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:18.112105 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfc4d3-49ac-432d-9f49-f9e369c2defa None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:18.113060 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dfc4d3-49ac-432d-9f49-f9e369c2defa None admin] RBAC: Adding query filter params (name=reader) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:18.113604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dfc4d3-49ac-432d-9f49-f9e369c2defa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:18.115472 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dfc4d3-49ac-432d-9f49-f9e369c2defa None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:18.127923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 168/346] 10.222.0.41 () {56 vars in 1101 bytes} [Fri Jun 3 16:02:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:18.129151 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dfc4d3-49ac-432d-9f49-f9e369c2defa None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:18.144611 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 179/347] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:02:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 16:02:18.146606 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6e9ae-9ad2-4693-99b8-ca3f9e7a04a3 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:18.146898 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6e9ae-9ad2-4693-99b8-ca3f9e7a04a3 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:18.147171 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6e9ae-9ad2-4693-99b8-ca3f9e7a04a3 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:18.148648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 180/348] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:18.149160 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d6e9ae-9ad2-4693-99b8-ca3f9e7a04a3 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:18.230247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:18.236513 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:18.268772 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dfc4d3-49ac-432d-9f49-f9e369c2defa None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:18.293018 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d6e9ae-9ad2-4693-99b8-ca3f9e7a04a3 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:18.313853 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716623c1-1b26-4bb0-b73f-fcb985da72ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:18.314521 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716623c1-1b26-4bb0-b73f-fcb985da72ff None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:18.314808 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716623c1-1b26-4bb0-b73f-fcb985da72ff None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:18.315084 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716623c1-1b26-4bb0-b73f-fcb985da72ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:18.331135 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b72d208-63b3-496c-9ec3-e4fbbe6dc079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:18.332313 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72d208-63b3-496c-9ec3-e4fbbe6dc079 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:18.333340 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72d208-63b3-496c-9ec3-e4fbbe6dc079 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:18.333769 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72d208-63b3-496c-9ec3-e4fbbe6dc079 None admin] PATH_INFO: `/v3/users/d652bf1b187f4d489009eceeee5c12ab` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:18.335173 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b72d208-63b3-496c-9ec3-e4fbbe6dc079 None admin] RBAC: Authorizing `identity:get_user(user_id=d652bf1b187f4d489009eceeee5c12ab)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:18.336919 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b72d208-63b3-496c-9ec3-e4fbbe6dc079 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:18.338793 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 181/349] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:02:18 2022] GET /identity/v3/users/d652bf1b187f4d489009eceeee5c12ab => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:18.341156 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde51905-125b-44be-b2f3-7cc10a05995e None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:18.341701 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde51905-125b-44be-b2f3-7cc10a05995e None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:18.342191 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde51905-125b-44be-b2f3-7cc10a05995e None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:18.347967 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 182/350] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:18.349615 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde51905-125b-44be-b2f3-7cc10a05995e None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:18.400242 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716623c1-1b26-4bb0-b73f-fcb985da72ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:18.401665 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716623c1-1b26-4bb0-b73f-fcb985da72ff None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:18.497788 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde51905-125b-44be-b2f3-7cc10a05995e None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:18.498650 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:18.540359 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 169/351] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:02:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 16:02:18.541716 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c338aeb-3130-4a10-bb5b-32835c806b13 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:18.542004 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c338aeb-3130-4a10-bb5b-32835c806b13 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:18.542281 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c338aeb-3130-4a10-bb5b-32835c806b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:18.545567 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:18.554180 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aa977c-321c-4cdf-8e55-76bbd65fa69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:18.555042 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa977c-321c-4cdf-8e55-76bbd65fa69e None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:18.555358 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa977c-321c-4cdf-8e55-76bbd65fa69e None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:18.555641 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa977c-321c-4cdf-8e55-76bbd65fa69e None admin] PATH_INFO: `/v3/projects/1d2895687ae2471682919de9b8fa4afb` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:18.556849 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aa977c-321c-4cdf-8e55-76bbd65fa69e None admin] RBAC: Authorizing `identity:get_project(project_id=1d2895687ae2471682919de9b8fa4afb)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:18.558286 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aa977c-321c-4cdf-8e55-76bbd65fa69e None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:18.560076 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 183/352] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:02:18 2022] GET /identity/v3/projects/1d2895687ae2471682919de9b8fa4afb => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:18.561725 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aa977c-321c-4cdf-8e55-76bbd65fa69e None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:18.568945 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-6c338aeb-3130-4a10-bb5b-32835c806b13 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:18.570142 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6c338aeb-3130-4a10-bb5b-32835c806b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:18.666815 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 170/353] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:18.668981 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd3cb8-7638-4fef-949e-e1ba310efd1d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:18.669467 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd3cb8-7638-4fef-949e-e1ba310efd1d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:18.670127 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd3cb8-7638-4fef-949e-e1ba310efd1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:18.691053 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aa977c-321c-4cdf-8e55-76bbd65fa69e None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:18.692789 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-5bcd3cb8-7638-4fef-949e-e1ba310efd1d None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:18.695267 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5bcd3cb8-7638-4fef-949e-e1ba310efd1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:18.739773 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b1f566-e1a5-4ec6-8e16-c5e8d8f63e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:18.740448 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b1f566-e1a5-4ec6-8e16-c5e8d8f63e4b None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:18.740732 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b1f566-e1a5-4ec6-8e16-c5e8d8f63e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:18.741009 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b1f566-e1a5-4ec6-8e16-c5e8d8f63e4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:18.741843 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b1f566-e1a5-4ec6-8e16-c5e8d8f63e4b None admin] RBAC: Adding query filter params (role.id=23e883fa91ba4c4f8fa46a1414e9761a, user.id=d652bf1b187f4d489009eceeee5c12ab, scope.system=all) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:18.742158 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b1f566-e1a5-4ec6-8e16-c5e8d8f63e4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:18.743648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b1f566-e1a5-4ec6-8e16-c5e8d8f63e4b None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:18.767351 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 184/354] 10.222.0.41 () {56 vars in 1308 bytes} [Fri Jun 3 16:02:18 2022] GET /identity/v3/role_assignments?user.id=d652bf1b187f4d489009eceeee5c12ab&scope.system=all&role.id=23e883fa91ba4c4f8fa46a1414e9761a => generated 510 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:18.767351 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b1f566-e1a5-4ec6-8e16-c5e8d8f63e4b None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:18.801239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 171/355] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:18.803418 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd3cb8-7638-4fef-949e-e1ba310efd1d None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:18.846766 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:18.852335 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:18.919444 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b1f566-e1a5-4ec6-8e16-c5e8d8f63e4b None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:18.940673 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd3cb8-7638-4fef-949e-e1ba310efd1d None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:18.963167 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21951989-c01d-4d38-a3a3-0d5142ba5cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:18.964101 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21951989-c01d-4d38-a3a3-0d5142ba5cae None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:18.964383 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21951989-c01d-4d38-a3a3-0d5142ba5cae None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:18.964655 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21951989-c01d-4d38-a3a3-0d5142ba5cae None admin] PATH_INFO: `/v3/projects/1d2895687ae2471682919de9b8fa4afb` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:18.965879 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21951989-c01d-4d38-a3a3-0d5142ba5cae None admin] RBAC: Authorizing `identity:get_project(project_id=1d2895687ae2471682919de9b8fa4afb)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:18.967764 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21951989-c01d-4d38-a3a3-0d5142ba5cae None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:18.969285 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 185/356] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:02:18 2022] GET /identity/v3/projects/1d2895687ae2471682919de9b8fa4afb => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:18.969786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21951989-c01d-4d38-a3a3-0d5142ba5cae None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:18.990670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf5af8-d5c2-4ff8-82c7-4d7faaa3f4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:18.992143 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf5af8-d5c2-4ff8-82c7-4d7faaa3f4ff None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:18.992431 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf5af8-d5c2-4ff8-82c7-4d7faaa3f4ff None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:18.992710 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf5af8-d5c2-4ff8-82c7-4d7faaa3f4ff None admin] PATH_INFO: `/v3/groups/ad54dac2940a42e194222806ee8b60e9` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:19.001111 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bf5af8-d5c2-4ff8-82c7-4d7faaa3f4ff None admin] RBAC: Authorizing `identity:get_group(group_id=ad54dac2940a42e194222806ee8b60e9)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:19.003401 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bf5af8-d5c2-4ff8-82c7-4d7faaa3f4ff None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:19.005807 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 172/357] 10.222.0.41 () {56 vars in 1180 bytes} [Fri Jun 3 16:02:18 2022] GET /identity/v3/groups/ad54dac2940a42e194222806ee8b60e9 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:19.006890 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331daf07-800e-4b14-be44-3402a84d130c None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:19.007177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331daf07-800e-4b14-be44-3402a84d130c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:19.007623 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331daf07-800e-4b14-be44-3402a84d130c None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:19.009172 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 173/358] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:19.010224 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331daf07-800e-4b14-be44-3402a84d130c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:19.108293 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21951989-c01d-4d38-a3a3-0d5142ba5cae None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:19.133447 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:19.134101 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:19.140362 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331daf07-800e-4b14-be44-3402a84d130c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:19.151554 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b28a65-d359-445e-a467-2a334b29f3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:19.152375 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b28a65-d359-445e-a467-2a334b29f3e0 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:19.152724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b28a65-d359-445e-a467-2a334b29f3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:19.153010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b28a65-d359-445e-a467-2a334b29f3e0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:19.165794 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b28a65-d359-445e-a467-2a334b29f3e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:19.177740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-50b28a65-d359-445e-a467-2a334b29f3e0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 16:02:19.177740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:19.177740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:19.177740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:19.177740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:19.177740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:19.177740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:19.177740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:19.177740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:19.177740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:19.177740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:19.177740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:19.177740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:19.177740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:19.177740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:19.177740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:19.179483 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:19.179483 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:19.179483 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:19.179483 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:19.179483 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:19.179483 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:19.179483 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:19.179483 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:19.179483 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:19.179483 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:19.179483 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:19.179483 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:19.179483 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:19.179483 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:19.179483 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:19.179483 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:19.180570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:19.180570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:19.180570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:19.180570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:19.180570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:19.180570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:19.180570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:19.180570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:19.180570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:19.180570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:19.180570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 16:02:19.180570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:19.180570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 186/359] 10.222.0.41 () {56 vars in 1114 bytes} [Fri Jun 3 16:02:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:19.180570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b28a65-d359-445e-a467-2a334b29f3e0 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:19.191885 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7201565-f7b8-4f35-808c-bcf10902ff64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:19.192977 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7201565-f7b8-4f35-808c-bcf10902ff64 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:19.193265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7201565-f7b8-4f35-808c-bcf10902ff64 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:19.193937 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7201565-f7b8-4f35-808c-bcf10902ff64 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:19.204903 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7201565-f7b8-4f35-808c-bcf10902ff64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:19.212672 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-e7201565-f7b8-4f35-808c-bcf10902ff64 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 16:02:19.212672 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:19.212672 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:19.212672 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:19.212672 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:19.212672 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:19.212672 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:19.212672 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:19.212672 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:19.212672 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:19.212672 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:19.212672 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:19.212672 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:19.212672 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:19.212672 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:19.212672 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:19.213688 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:19.213688 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:19.213688 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:19.213688 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:19.213688 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:19.213688 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:19.213688 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:19.213688 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:19.213688 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:19.213688 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:19.213688 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:19.213688 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:19.213688 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:19.213688 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:19.213688 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:19.213688 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:19.214985 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:19.214985 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:19.214985 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:19.214985 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:19.214985 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:19.214985 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:19.214985 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:19.214985 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:19.214985 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:19.214985 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:19.214985 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 16:02:19.214985 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:19.214985 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 174/360] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:02:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:19.214985 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7201565-f7b8-4f35-808c-bcf10902ff64 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:19.302842 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b28a65-d359-445e-a467-2a334b29f3e0 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:19.340061 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7201565-f7b8-4f35-808c-bcf10902ff64 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:19.341104 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00388f6a-b6a1-4392-8042-5722dff3a629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:19.341999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00388f6a-b6a1-4392-8042-5722dff3a629 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:19.342290 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00388f6a-b6a1-4392-8042-5722dff3a629 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:19.342565 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00388f6a-b6a1-4392-8042-5722dff3a629 None admin] PATH_INFO: `/v3/groups/ad54dac2940a42e194222806ee8b60e9` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:19.344098 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00388f6a-b6a1-4392-8042-5722dff3a629 None admin] RBAC: Authorizing `identity:get_group(group_id=ad54dac2940a42e194222806ee8b60e9)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:19.345540 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00388f6a-b6a1-4392-8042-5722dff3a629 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:19.347457 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 187/361] 10.222.0.41 () {56 vars in 1180 bytes} [Fri Jun 3 16:02:19 2022] GET /identity/v3/groups/ad54dac2940a42e194222806ee8b60e9 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:19.348962 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179cb6a8-7e56-42dc-8d08-43a65cac9847 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:19.349247 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179cb6a8-7e56-42dc-8d08-43a65cac9847 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:19.349520 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179cb6a8-7e56-42dc-8d08-43a65cac9847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:19.370038 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-179cb6a8-7e56-42dc-8d08-43a65cac9847 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:19.371769 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-179cb6a8-7e56-42dc-8d08-43a65cac9847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:19.384589 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28f95a-1c1c-404e-854d-4293de15eba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:19.385387 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28f95a-1c1c-404e-854d-4293de15eba9 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:19.385506 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28f95a-1c1c-404e-854d-4293de15eba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:19.385774 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28f95a-1c1c-404e-854d-4293de15eba9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:19.386608 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b28f95a-1c1c-404e-854d-4293de15eba9 None admin] RBAC: Adding query filter params (group.id=ad54dac2940a42e194222806ee8b60e9, role.id=2248ac4d70c34e459b8942dec0d7656b, scope.project.id=1d2895687ae2471682919de9b8fa4afb) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:19.386923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b28f95a-1c1c-404e-854d-4293de15eba9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:19.391096 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b28f95a-1c1c-404e-854d-4293de15eba9 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:19.396106 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 175/362] 10.222.0.41 () {56 vars in 1376 bytes} [Fri Jun 3 16:02:19 2022] GET /identity/v3/role_assignments?group.id=ad54dac2940a42e194222806ee8b60e9&scope.project.id=1d2895687ae2471682919de9b8fa4afb&role.id=2248ac4d70c34e459b8942dec0d7656b => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:19.396929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28f95a-1c1c-404e-854d-4293de15eba9 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:19.435703 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:19.440173 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:19.470846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 188/363] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:19.472158 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179cb6a8-7e56-42dc-8d08-43a65cac9847 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:19.542492 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28f95a-1c1c-404e-854d-4293de15eba9 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:19.595003 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ffc137-bf91-4067-8a68-742b5d060640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:19.596962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ffc137-bf91-4067-8a68-742b5d060640 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:19.596962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ffc137-bf91-4067-8a68-742b5d060640 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:19.596962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ffc137-bf91-4067-8a68-742b5d060640 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:19.596962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ffc137-bf91-4067-8a68-742b5d060640 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:19.597307 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ffc137-bf91-4067-8a68-742b5d060640 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:19.599207 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ffc137-bf91-4067-8a68-742b5d060640 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:19.611339 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 176/364] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:02:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:19.612778 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ffc137-bf91-4067-8a68-742b5d060640 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:19.626856 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179cb6a8-7e56-42dc-8d08-43a65cac9847 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:19.664701 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b6d335-c064-4a56-b0d1-9fbab7647e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:19.665380 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6d335-c064-4a56-b0d1-9fbab7647e31 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:19.665662 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6d335-c064-4a56-b0d1-9fbab7647e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:19.665939 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6d335-c064-4a56-b0d1-9fbab7647e31 None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:19.666811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b6d335-c064-4a56-b0d1-9fbab7647e31 None admin] RBAC: Adding query filter params (name=member) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:19.667118 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b6d335-c064-4a56-b0d1-9fbab7647e31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:19.668540 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b6d335-c064-4a56-b0d1-9fbab7647e31 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:19.680187 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 189/365] 10.222.0.41 () {56 vars in 1101 bytes} [Fri Jun 3 16:02:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:19.681146 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b6d335-c064-4a56-b0d1-9fbab7647e31 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:19.712918 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:19.730059 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ffc137-bf91-4067-8a68-742b5d060640 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:19.740902 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:19.778239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c5afdf-b271-4de4-b4b5-1b3fce0a1dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:19.778900 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5afdf-b271-4de4-b4b5-1b3fce0a1dcf None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:19.779470 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5afdf-b271-4de4-b4b5-1b3fce0a1dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:19.779749 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5afdf-b271-4de4-b4b5-1b3fce0a1dcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:19.780651 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c5afdf-b271-4de4-b4b5-1b3fce0a1dcf None admin] RBAC: Adding query filter params (group.id=ad54dac2940a42e194222806ee8b60e9, role.id=a8e4174e3572497f9284c709a119dcff, scope.project.id=1d2895687ae2471682919de9b8fa4afb) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:19.780991 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c5afdf-b271-4de4-b4b5-1b3fce0a1dcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:19.782445 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c5afdf-b271-4de4-b4b5-1b3fce0a1dcf None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:19.789871 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 177/366] 10.222.0.41 () {56 vars in 1376 bytes} [Fri Jun 3 16:02:19 2022] GET /identity/v3/role_assignments?group.id=ad54dac2940a42e194222806ee8b60e9&scope.project.id=1d2895687ae2471682919de9b8fa4afb&role.id=a8e4174e3572497f9284c709a119dcff => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:19.790679 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c5afdf-b271-4de4-b4b5-1b3fce0a1dcf None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:19.813392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b6d335-c064-4a56-b0d1-9fbab7647e31 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:19.854274 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6dfa59-a8ec-4244-945f-5f8e4a972b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:19.855084 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6dfa59-a8ec-4244-945f-5f8e4a972b29 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:19.855415 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6dfa59-a8ec-4244-945f-5f8e4a972b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:19.855690 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6dfa59-a8ec-4244-945f-5f8e4a972b29 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:19.868120 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6dfa59-a8ec-4244-945f-5f8e4a972b29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:19.876417 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-3e6dfa59-a8ec-4244-945f-5f8e4a972b29 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:02:19.876417 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:19.876417 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:19.876417 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:19.876417 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:19.876417 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:19.876417 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:19.876417 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:19.876417 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:19.876417 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:19.876417 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:19.876417 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:19.876417 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:19.876417 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:19.876417 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:19.876417 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:19.877392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:19.877392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:19.877392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:19.877392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:19.877392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:19.877392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:19.877392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:19.877392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:19.877392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:19.877392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:19.877392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:19.877392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:19.877392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:19.877392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:19.877392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:19.877392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:19.878172 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:19.878172 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:19.878172 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:19.878172 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:19.878172 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:19.878172 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:19.878172 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:19.878172 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:19.878172 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:19.878172 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:19.878172 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:02:19.878172 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:19.878172 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 190/367] 10.222.0.41 () {56 vars in 1096 bytes} [Fri Jun 3 16:02:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:19.878831 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6dfa59-a8ec-4244-945f-5f8e4a972b29 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:19.946493 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c5afdf-b271-4de4-b4b5-1b3fce0a1dcf None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:19.993380 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6472654-c4b0-4bd9-b6c5-b9f151dbdaee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:19.993960 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6472654-c4b0-4bd9-b6c5-b9f151dbdaee None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:19.994244 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6472654-c4b0-4bd9-b6c5-b9f151dbdaee None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:19.994520 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6472654-c4b0-4bd9-b6c5-b9f151dbdaee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:20.014937 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6dfa59-a8ec-4244-945f-5f8e4a972b29 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:20.018428 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:20.021798 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:20.059030 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ebe82-38e7-40da-84cd-4cbb53f57ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:20.060182 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ebe82-38e7-40da-84cd-4cbb53f57ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:20.060660 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ebe82-38e7-40da-84cd-4cbb53f57ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:20.061141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ebe82-38e7-40da-84cd-4cbb53f57ad9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:20.097221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6472654-c4b0-4bd9-b6c5-b9f151dbdaee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:20.099582 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6472654-c4b0-4bd9-b6c5-b9f151dbdaee None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:20.162109 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ebe82-38e7-40da-84cd-4cbb53f57ad9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:20.164589 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ebe82-38e7-40da-84cd-4cbb53f57ad9 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:20.240230 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 178/368] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:02:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 16:02:20.241470 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6472654-c4b0-4bd9-b6c5-b9f151dbdaee None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:20.283390 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 191/369] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:02:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 16:02:20.284251 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ebe82-38e7-40da-84cd-4cbb53f57ad9 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:20.287423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:20.303450 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:20.360558 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6472654-c4b0-4bd9-b6c5-b9f151dbdaee None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:20.390993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ebe82-38e7-40da-84cd-4cbb53f57ad9 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:20.397429 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073ebe8a-6b88-44c9-8693-ac6b23a2364a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:20.398094 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ebe8a-6b88-44c9-8693-ac6b23a2364a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:20.398377 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ebe8a-6b88-44c9-8693-ac6b23a2364a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:20.398653 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ebe8a-6b88-44c9-8693-ac6b23a2364a None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:20.399430 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073ebe8a-6b88-44c9-8693-ac6b23a2364a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:20.399740 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073ebe8a-6b88-44c9-8693-ac6b23a2364a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:20.401219 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073ebe8a-6b88-44c9-8693-ac6b23a2364a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:20.410513 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 179/370] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:02:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:20.411397 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073ebe8a-6b88-44c9-8693-ac6b23a2364a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:20.430214 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f38a2-e6e5-4d98-a374-88bc3069f498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:20.431422 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f38a2-e6e5-4d98-a374-88bc3069f498 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:20.431886 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f38a2-e6e5-4d98-a374-88bc3069f498 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:20.432335 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f38a2-e6e5-4d98-a374-88bc3069f498 None admin] PATH_INFO: `/v3/projects/23d03cc631264a3faa207c1b3bded8e3` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:20.433713 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989f38a2-e6e5-4d98-a374-88bc3069f498 None admin] RBAC: Authorizing `identity:get_project(project_id=23d03cc631264a3faa207c1b3bded8e3)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:20.435341 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989f38a2-e6e5-4d98-a374-88bc3069f498 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:20.437230 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 192/371] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:02:20 2022] GET /identity/v3/projects/23d03cc631264a3faa207c1b3bded8e3 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:20.438176 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f38a2-e6e5-4d98-a374-88bc3069f498 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:20.511416 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:20.518267 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073ebe8a-6b88-44c9-8693-ac6b23a2364a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:20.533102 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:20.549671 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f38a2-e6e5-4d98-a374-88bc3069f498 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:20.555721 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff00e42-a6ee-414a-82f8-2d96f3c020d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:20.556589 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff00e42-a6ee-414a-82f8-2d96f3c020d3 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:20.556870 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff00e42-a6ee-414a-82f8-2d96f3c020d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:20.557145 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff00e42-a6ee-414a-82f8-2d96f3c020d3 None admin] PATH_INFO: `/v3/projects/23d03cc631264a3faa207c1b3bded8e3` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:20.558429 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff00e42-a6ee-414a-82f8-2d96f3c020d3 None admin] RBAC: Authorizing `identity:get_project(project_id=23d03cc631264a3faa207c1b3bded8e3)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:20.560130 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff00e42-a6ee-414a-82f8-2d96f3c020d3 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:20.562130 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 180/372] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:02:20 2022] GET /identity/v3/projects/23d03cc631264a3faa207c1b3bded8e3 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:20.562910 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff00e42-a6ee-414a-82f8-2d96f3c020d3 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:20.589368 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51577c84-1d6f-4a1e-a18a-cd0e8801fb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:20.590314 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51577c84-1d6f-4a1e-a18a-cd0e8801fb16 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:20.590800 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51577c84-1d6f-4a1e-a18a-cd0e8801fb16 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:20.591265 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51577c84-1d6f-4a1e-a18a-cd0e8801fb16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:20.668394 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51577c84-1d6f-4a1e-a18a-cd0e8801fb16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:20.670217 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51577c84-1d6f-4a1e-a18a-cd0e8801fb16 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:20.674916 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff00e42-a6ee-414a-82f8-2d96f3c020d3 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:20.713496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36dff17-37dc-4bff-95c0-fbdecffdd07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:20.714801 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36dff17-37dc-4bff-95c0-fbdecffdd07d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:20.714801 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36dff17-37dc-4bff-95c0-fbdecffdd07d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:20.714987 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36dff17-37dc-4bff-95c0-fbdecffdd07d None admin] PATH_INFO: `/v3/groups/ad54dac2940a42e194222806ee8b60e9` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:20.716085 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36dff17-37dc-4bff-95c0-fbdecffdd07d None admin] RBAC: Authorizing `identity:get_group(group_id=ad54dac2940a42e194222806ee8b60e9)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:20.717549 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36dff17-37dc-4bff-95c0-fbdecffdd07d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:20.719649 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 181/373] 10.222.0.41 () {56 vars in 1180 bytes} [Fri Jun 3 16:02:20 2022] GET /identity/v3/groups/ad54dac2940a42e194222806ee8b60e9 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:20.720622 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36dff17-37dc-4bff-95c0-fbdecffdd07d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:20.769737 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:20.779697 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:20.792149 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 193/374] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:02:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 16:02:20.793610 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51577c84-1d6f-4a1e-a18a-cd0e8801fb16 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:20.828384 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36dff17-37dc-4bff-95c0-fbdecffdd07d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:20.866371 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692ac07-d63d-4af0-8996-5a30e1c24858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:20.867327 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692ac07-d63d-4af0-8996-5a30e1c24858 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:20.867566 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692ac07-d63d-4af0-8996-5a30e1c24858 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:20.867842 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692ac07-d63d-4af0-8996-5a30e1c24858 None admin] PATH_INFO: `/v3/groups/ad54dac2940a42e194222806ee8b60e9` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:20.869070 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b692ac07-d63d-4af0-8996-5a30e1c24858 None admin] RBAC: Authorizing `identity:get_group(group_id=ad54dac2940a42e194222806ee8b60e9)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:20.870993 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b692ac07-d63d-4af0-8996-5a30e1c24858 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:20.872220 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 182/375] 10.222.0.41 () {56 vars in 1180 bytes} [Fri Jun 3 16:02:20 2022] GET /identity/v3/groups/ad54dac2940a42e194222806ee8b60e9 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:20.872971 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692ac07-d63d-4af0-8996-5a30e1c24858 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:20.905223 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51577c84-1d6f-4a1e-a18a-cd0e8801fb16 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:20.942776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322af52f-9b14-4b57-9faf-9700af8c2668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:20.943799 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322af52f-9b14-4b57-9faf-9700af8c2668 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:20.944307 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322af52f-9b14-4b57-9faf-9700af8c2668 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:20.944794 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322af52f-9b14-4b57-9faf-9700af8c2668 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:20.946086 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322af52f-9b14-4b57-9faf-9700af8c2668 None admin] RBAC: Adding query filter params (group.id=ad54dac2940a42e194222806ee8b60e9, role.id=2248ac4d70c34e459b8942dec0d7656b, scope.project.id=23d03cc631264a3faa207c1b3bded8e3) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:20.946650 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322af52f-9b14-4b57-9faf-9700af8c2668 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:20.948317 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322af52f-9b14-4b57-9faf-9700af8c2668 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:20.956538 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 194/376] 10.222.0.41 () {56 vars in 1376 bytes} [Fri Jun 3 16:02:20 2022] GET /identity/v3/role_assignments?group.id=ad54dac2940a42e194222806ee8b60e9&scope.project.id=23d03cc631264a3faa207c1b3bded8e3&role.id=2248ac4d70c34e459b8942dec0d7656b => generated 260 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:20.958394 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322af52f-9b14-4b57-9faf-9700af8c2668 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:20.983409 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692ac07-d63d-4af0-8996-5a30e1c24858 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:21.010558 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:21.031818 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:21.044895 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465a9dd-09a9-4efc-b63b-a33a5818ce31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:21.045860 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465a9dd-09a9-4efc-b63b-a33a5818ce31 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:21.046141 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465a9dd-09a9-4efc-b63b-a33a5818ce31 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:21.046400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465a9dd-09a9-4efc-b63b-a33a5818ce31 None admin] PATH_INFO: `/v3/projects/450b64d0f10f4cb8949a26d9eea4aef4` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:21.047802 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3465a9dd-09a9-4efc-b63b-a33a5818ce31 None admin] RBAC: Authorizing `identity:get_project(project_id=450b64d0f10f4cb8949a26d9eea4aef4)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:21.049609 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3465a9dd-09a9-4efc-b63b-a33a5818ce31 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:21.051600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 183/377] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:02:20 2022] GET /identity/v3/projects/450b64d0f10f4cb8949a26d9eea4aef4 => generated 344 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:21.056732 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465a9dd-09a9-4efc-b63b-a33a5818ce31 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:21.088396 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322af52f-9b14-4b57-9faf-9700af8c2668 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:21.173419 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec7761-404c-4f7f-a120-c48461a0b278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:21.175044 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec7761-404c-4f7f-a120-c48461a0b278 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:21.175044 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec7761-404c-4f7f-a120-c48461a0b278 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:21.175044 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec7761-404c-4f7f-a120-c48461a0b278 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:21.183960 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ec7761-404c-4f7f-a120-c48461a0b278 None admin] RBAC: Adding query filter params (group.id=ad54dac2940a42e194222806ee8b60e9, role.id=a8e4174e3572497f9284c709a119dcff, scope.project.id=23d03cc631264a3faa207c1b3bded8e3) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:21.183960 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ec7761-404c-4f7f-a120-c48461a0b278 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:21.185471 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ec7761-404c-4f7f-a120-c48461a0b278 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:21.191700 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 195/378] 10.222.0.41 () {56 vars in 1376 bytes} [Fri Jun 3 16:02:20 2022] GET /identity/v3/role_assignments?group.id=ad54dac2940a42e194222806ee8b60e9&scope.project.id=23d03cc631264a3faa207c1b3bded8e3&role.id=a8e4174e3572497f9284c709a119dcff => generated 260 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:21.198921 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465a9dd-09a9-4efc-b63b-a33a5818ce31 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:21.258180 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7aec07-1741-4344-aa82-58673251d615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:21.259063 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7aec07-1741-4344-aa82-58673251d615 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:21.259367 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7aec07-1741-4344-aa82-58673251d615 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:21.259650 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7aec07-1741-4344-aa82-58673251d615 None admin] PATH_INFO: `/v3/groups/06ee27f162d145e79244190e976f5e0d` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:21.267374 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7aec07-1741-4344-aa82-58673251d615 None admin] RBAC: Authorizing `identity:get_group(group_id=06ee27f162d145e79244190e976f5e0d)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:21.268867 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7aec07-1741-4344-aa82-58673251d615 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:21.270981 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 184/379] 10.222.0.41 () {56 vars in 1180 bytes} [Fri Jun 3 16:02:21 2022] GET /identity/v3/groups/06ee27f162d145e79244190e976f5e0d => generated 229 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:02:21.276731 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec7761-404c-4f7f-a120-c48461a0b278 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:21.398900 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:21.401776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec7761-404c-4f7f-a120-c48461a0b278 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:21.440592 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3f5b9-f33e-448c-9695-9788b8adbe23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:21.441259 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3f5b9-f33e-448c-9695-9788b8adbe23 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:21.441541 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3f5b9-f33e-448c-9695-9788b8adbe23 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:21.441816 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3f5b9-f33e-448c-9695-9788b8adbe23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:21.443244 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d3f5b9-f33e-448c-9695-9788b8adbe23 None admin] RBAC: Adding query filter params (group.id=06ee27f162d145e79244190e976f5e0d, role.id=75936a608b8149099ff54e317f3a6740, scope.project.id=450b64d0f10f4cb8949a26d9eea4aef4) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:21.443580 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d3f5b9-f33e-448c-9695-9788b8adbe23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:21.445032 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d3f5b9-f33e-448c-9695-9788b8adbe23 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:21.451724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 196/380] 10.222.0.41 () {56 vars in 1376 bytes} [Fri Jun 3 16:02:21 2022] GET /identity/v3/role_assignments?group.id=06ee27f162d145e79244190e976f5e0d&scope.project.id=450b64d0f10f4cb8949a26d9eea4aef4&role.id=75936a608b8149099ff54e317f3a6740 => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:22.124626 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c059839-d597-43bb-a2ce-1afdf72cea9e None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:22.125649 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c059839-d597-43bb-a2ce-1afdf72cea9e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:22.126141 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c059839-d597-43bb-a2ce-1afdf72cea9e None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:22.128013 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 185/381] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:22.133202 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ccd2f-79e0-4d56-81a6-fded30e6ed05 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:22.133628 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ccd2f-79e0-4d56-81a6-fded30e6ed05 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:22.133802 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ccd2f-79e0-4d56-81a6-fded30e6ed05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:22.155432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-f42ccd2f-79e0-4d56-81a6-fded30e6ed05 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:22.156607 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-f42ccd2f-79e0-4d56-81a6-fded30e6ed05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:22.246578 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 197/382] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:22.284590 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c059839-d597-43bb-a2ce-1afdf72cea9e None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:22.360343 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:22.437010 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c059839-d597-43bb-a2ce-1afdf72cea9e None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:22.486772 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eeb83c-8e3d-4688-9809-54fa5c2d6926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:22.487623 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eeb83c-8e3d-4688-9809-54fa5c2d6926 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:22.487914 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eeb83c-8e3d-4688-9809-54fa5c2d6926 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:22.488195 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eeb83c-8e3d-4688-9809-54fa5c2d6926 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:22.504159 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eeb83c-8e3d-4688-9809-54fa5c2d6926 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:22.512928 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-e7eeb83c-8e3d-4688-9809-54fa5c2d6926 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 16:02:22.512928 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:22.512928 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:22.512928 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:22.512928 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:22.512928 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:22.512928 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:22.512928 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:22.512928 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:22.512928 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:22.512928 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:22.512928 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:22.512928 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:22.512928 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:22.512928 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:22.512928 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:22.513814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:22.513814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:22.513814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:22.513814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:22.513814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:22.513814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:22.513814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:22.513814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:22.513814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:22.513814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:22.513814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:22.513814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:22.513814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:22.513814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:22.513814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:22.513814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:22.514648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:22.514648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:22.514648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:22.514648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:22.514648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:22.514648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:22.514648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:22.514648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:22.514648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:22.514648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:22.514648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 16:02:22.514648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:22.518727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 186/383] 10.222.0.41 () {56 vars in 1114 bytes} [Fri Jun 3 16:02:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:22.520928 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ccd2f-79e0-4d56-81a6-fded30e6ed05 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:22.613761 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e891b-c9ba-446c-b1f5-6a9cdeef1448 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:22.614085 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e891b-c9ba-446c-b1f5-6a9cdeef1448 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:22.614339 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e891b-c9ba-446c-b1f5-6a9cdeef1448 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:22.616037 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 187/384] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 16:02:22.636744 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16493b28-3ed3-493a-9a41-facbd67ca2ab None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:22.636744 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16493b28-3ed3-493a-9a41-facbd67ca2ab None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:22.636928 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16493b28-3ed3-493a-9a41-facbd67ca2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:22.666096 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:22.673398 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ccd2f-79e0-4d56-81a6-fded30e6ed05 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:22.676418 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-16493b28-3ed3-493a-9a41-facbd67ca2ab None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:22.677592 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-16493b28-3ed3-493a-9a41-facbd67ca2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:22.711399 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff76ecf-0f26-4a82-afa6-a1654db3189f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:22.712327 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff76ecf-0f26-4a82-afa6-a1654db3189f None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:22.712794 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff76ecf-0f26-4a82-afa6-a1654db3189f None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:22.713248 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff76ecf-0f26-4a82-afa6-a1654db3189f None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:22.714156 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff76ecf-0f26-4a82-afa6-a1654db3189f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:22.714640 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff76ecf-0f26-4a82-afa6-a1654db3189f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:22.716571 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff76ecf-0f26-4a82-afa6-a1654db3189f None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:22.725647 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 198/385] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:02:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:22.731733 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff76ecf-0f26-4a82-afa6-a1654db3189f None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:22.835335 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff76ecf-0f26-4a82-afa6-a1654db3189f None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:22.846126 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 188/386] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:22.892291 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b993a0-327f-49ba-a376-913722738152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:22.893500 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b993a0-327f-49ba-a376-913722738152 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:22.893954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b993a0-327f-49ba-a376-913722738152 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:22.894410 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b993a0-327f-49ba-a376-913722738152 None admin] PATH_INFO: `/v3/groups/ad54dac2940a42e194222806ee8b60e9` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:22.897772 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b993a0-327f-49ba-a376-913722738152 None admin] RBAC: Authorizing `identity:get_group(group_id=ad54dac2940a42e194222806ee8b60e9)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:22.898345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16493b28-3ed3-493a-9a41-facbd67ca2ab None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:22.899609 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b993a0-327f-49ba-a376-913722738152 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:22.901704 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 199/387] 10.222.0.41 () {56 vars in 1180 bytes} [Fri Jun 3 16:02:22 2022] GET /identity/v3/groups/ad54dac2940a42e194222806ee8b60e9 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:02:22.910054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b993a0-327f-49ba-a376-913722738152 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:22.964276 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:22.967653 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:23.086071 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b993a0-327f-49ba-a376-913722738152 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:23.101401 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16493b28-3ed3-493a-9a41-facbd67ca2ab None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:23.125152 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05773f-28bf-43f8-943b-7e78e14f9bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:23.126024 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05773f-28bf-43f8-943b-7e78e14f9bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:23.126306 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05773f-28bf-43f8-943b-7e78e14f9bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:23.126578 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05773f-28bf-43f8-943b-7e78e14f9bf2 None admin] PATH_INFO: `/v3/projects/1d2895687ae2471682919de9b8fa4afb` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:23.128402 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05773f-28bf-43f8-943b-7e78e14f9bf2 None admin] RBAC: Authorizing `identity:get_project(project_id=1d2895687ae2471682919de9b8fa4afb)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:23.129863 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05773f-28bf-43f8-943b-7e78e14f9bf2 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:23.134320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 200/388] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:02:22 2022] GET /identity/v3/projects/1d2895687ae2471682919de9b8fa4afb => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:23.140339 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05773f-28bf-43f8-943b-7e78e14f9bf2 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:23.180218 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f6fdcc-4aa0-4681-b682-2f9aef66788b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:23.181011 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6fdcc-4aa0-4681-b682-2f9aef66788b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:23.181351 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6fdcc-4aa0-4681-b682-2f9aef66788b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:23.181623 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6fdcc-4aa0-4681-b682-2f9aef66788b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:23.204219 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f6fdcc-4aa0-4681-b682-2f9aef66788b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:23.212020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-17f6fdcc-4aa0-4681-b682-2f9aef66788b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 16:02:23.212020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:23.212020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:23.212020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:23.212020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:23.212020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:23.212020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:23.212020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:23.212020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:23.212020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:23.212020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:23.212020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:23.212020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:23.212020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:23.212020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:23.212020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:23.212917 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:23.212917 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:23.212917 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:23.212917 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:23.212917 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:23.212917 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:23.212917 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:23.212917 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:23.212917 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:23.212917 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:23.212917 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:23.212917 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:23.212917 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:23.212917 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:23.212917 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:23.212917 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:23.213726 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:23.213726 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:23.213726 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:23.213726 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:23.213726 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:23.213726 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:23.213726 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:23.213726 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:23.213726 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:23.213726 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:23.213726 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 16:02:23.213726 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:23.213726 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 189/389] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:02:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:02:23.221033 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f6fdcc-4aa0-4681-b682-2f9aef66788b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:23.306647 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05773f-28bf-43f8-943b-7e78e14f9bf2 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:23.331666 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:23.348618 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26395baa-3d20-4c96-960f-5159c32428d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:23.349554 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26395baa-3d20-4c96-960f-5159c32428d0 None admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:23.349838 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26395baa-3d20-4c96-960f-5159c32428d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:23.350109 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26395baa-3d20-4c96-960f-5159c32428d0 None admin] PATH_INFO: `/v3/projects/1d2895687ae2471682919de9b8fa4afb/groups/ad54dac2940a42e194222806ee8b60e9/roles/2248ac4d70c34e459b8942dec0d7656b` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:23.352550 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26395baa-3d20-4c96-960f-5159c32428d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d2895687ae2471682919de9b8fa4afb, group_id=ad54dac2940a42e194222806ee8b60e9, role_id=2248ac4d70c34e459b8942dec0d7656b)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:23.354846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26395baa-3d20-4c96-960f-5159c32428d0 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:23.374270 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 201/390] 10.222.0.41 () {58 vars in 1442 bytes} [Fri Jun 3 16:02:23 2022] PUT /identity/v3/projects/1d2895687ae2471682919de9b8fa4afb/groups/ad54dac2940a42e194222806ee8b60e9/roles/2248ac4d70c34e459b8942dec0d7656b => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:02:23.389781 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:23.398483 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f6fdcc-4aa0-4681-b682-2f9aef66788b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:23.444690 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f80dcf9-5c96-4e07-9bbf-b2cfde9e37b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:23.445368 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80dcf9-5c96-4e07-9bbf-b2cfde9e37b3 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:23.445697 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80dcf9-5c96-4e07-9bbf-b2cfde9e37b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:23.446079 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80dcf9-5c96-4e07-9bbf-b2cfde9e37b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:23.447149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f80dcf9-5c96-4e07-9bbf-b2cfde9e37b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:23.447643 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f80dcf9-5c96-4e07-9bbf-b2cfde9e37b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:23.449708 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f80dcf9-5c96-4e07-9bbf-b2cfde9e37b3 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:23.469691 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 190/391] 10.222.0.41 () {56 vars in 1101 bytes} [Fri Jun 3 16:02:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:23.475400 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26395baa-3d20-4c96-960f-5159c32428d0 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:23.608896 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26395baa-3d20-4c96-960f-5159c32428d0 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:23.619146 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c69121-cbab-4e6d-b20b-3ea35d75d1b9 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:23.619810 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c69121-cbab-4e6d-b20b-3ea35d75d1b9 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:23.620366 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c69121-cbab-4e6d-b20b-3ea35d75d1b9 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:23.622254 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 191/392] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:23.628142 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a974fb9-d434-4caa-9fae-24094efb7cc8 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:23.628733 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a974fb9-d434-4caa-9fae-24094efb7cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:23.629212 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a974fb9-d434-4caa-9fae-24094efb7cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:23.648902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-1a974fb9-d434-4caa-9fae-24094efb7cc8 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:23.650551 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1a974fb9-d434-4caa-9fae-24094efb7cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:23.672657 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6cd96-9dbc-48ce-af6e-4b9b4de1b6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:23.672657 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6cd96-9dbc-48ce-af6e-4b9b4de1b6d8 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:23.672657 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6cd96-9dbc-48ce-af6e-4b9b4de1b6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:23.672657 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6cd96-9dbc-48ce-af6e-4b9b4de1b6d8 None admin] PATH_INFO: `/v3/groups/ad54dac2940a42e194222806ee8b60e9` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:23.673080 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b6cd96-9dbc-48ce-af6e-4b9b4de1b6d8 None admin] RBAC: Authorizing `identity:get_group(group_id=ad54dac2940a42e194222806ee8b60e9)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:23.673702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b6cd96-9dbc-48ce-af6e-4b9b4de1b6d8 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:23.679326 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 202/393] 10.222.0.41 () {56 vars in 1180 bytes} [Fri Jun 3 16:02:23 2022] GET /identity/v3/groups/ad54dac2940a42e194222806ee8b60e9 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:23.682228 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6cd96-9dbc-48ce-af6e-4b9b4de1b6d8 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:23.734626 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:23.766977 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 192/394] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 16:02:23.794665 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a974fb9-d434-4caa-9fae-24094efb7cc8 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:23.840234 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6cd96-9dbc-48ce-af6e-4b9b4de1b6d8 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:23.876469 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7668b5c1-f039-4ba3-98ca-2ec8aba5f13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:23.877336 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668b5c1-f039-4ba3-98ca-2ec8aba5f13f None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:23.877619 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668b5c1-f039-4ba3-98ca-2ec8aba5f13f None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:23.877896 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668b5c1-f039-4ba3-98ca-2ec8aba5f13f None admin] PATH_INFO: `/v3/projects/1d2895687ae2471682919de9b8fa4afb` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:23.880427 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7668b5c1-f039-4ba3-98ca-2ec8aba5f13f None admin] RBAC: Authorizing `identity:get_project(project_id=1d2895687ae2471682919de9b8fa4afb)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:23.881866 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7668b5c1-f039-4ba3-98ca-2ec8aba5f13f None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:23.887531 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 203/395] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:02:23 2022] GET /identity/v3/projects/1d2895687ae2471682919de9b8fa4afb => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:23.889872 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7668b5c1-f039-4ba3-98ca-2ec8aba5f13f None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:23.909190 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a974fb9-d434-4caa-9fae-24094efb7cc8 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:23.939711 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:23.948647 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d78107-0357-4283-b0a3-a8f74535ae3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:23.949467 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d78107-0357-4283-b0a3-a8f74535ae3e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:23.950363 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d78107-0357-4283-b0a3-a8f74535ae3e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:23.950363 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d78107-0357-4283-b0a3-a8f74535ae3e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:23.963336 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d78107-0357-4283-b0a3-a8f74535ae3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:23.972084 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-33d78107-0357-4283-b0a3-a8f74535ae3e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 16:02:23.972084 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:23.972084 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:23.972084 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:23.972084 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:23.972084 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:23.972084 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:23.972084 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:23.972084 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:23.972084 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:23.972084 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:23.972084 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:23.972084 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:23.972084 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:23.972084 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:23.972084 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:23.973202 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:23.973202 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:23.973202 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:23.973202 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:23.973202 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:23.973202 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:23.973202 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:23.973202 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:23.973202 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:23.973202 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:23.973202 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:23.973202 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:23.973202 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:23.973202 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:23.973202 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:23.973202 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:23.974604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:23.974604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:23.974604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:23.974604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:23.974604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:23.974604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:23.974604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:23.974604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:23.974604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:23.974604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:23.974604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 16:02:23.974604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:23.974604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 193/396] 10.222.0.41 () {56 vars in 1114 bytes} [Fri Jun 3 16:02:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:02:23.985807 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d78107-0357-4283-b0a3-a8f74535ae3e None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:23.985966 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:23.997640 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7668b5c1-f039-4ba3-98ca-2ec8aba5f13f None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:24.038936 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5831e001-150a-44c5-a2f5-8333e1d571a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:24.039873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831e001-150a-44c5-a2f5-8333e1d571a9 None admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:24.040162 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831e001-150a-44c5-a2f5-8333e1d571a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:24.040443 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831e001-150a-44c5-a2f5-8333e1d571a9 None admin] PATH_INFO: `/v3/projects/1d2895687ae2471682919de9b8fa4afb/groups/ad54dac2940a42e194222806ee8b60e9/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:24.043595 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5831e001-150a-44c5-a2f5-8333e1d571a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d2895687ae2471682919de9b8fa4afb, group_id=ad54dac2940a42e194222806ee8b60e9, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:24.045029 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5831e001-150a-44c5-a2f5-8333e1d571a9 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:24.066084 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 204/397] 10.222.0.41 () {58 vars in 1442 bytes} [Fri Jun 3 16:02:23 2022] PUT /identity/v3/projects/1d2895687ae2471682919de9b8fa4afb/groups/ad54dac2940a42e194222806ee8b60e9/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:02:24.067930 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26301a-8191-415f-8d83-efc954d32163 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:24.068217 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26301a-8191-415f-8d83-efc954d32163 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:24.068490 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26301a-8191-415f-8d83-efc954d32163 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:24.069993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 205/398] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:24.092072 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571fb2e-b805-49aa-bab7-2e78868a4e1d None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:24.092429 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571fb2e-b805-49aa-bab7-2e78868a4e1d None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:24.092727 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571fb2e-b805-49aa-bab7-2e78868a4e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:24.111565 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-e571fb2e-b805-49aa-bab7-2e78868a4e1d None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:24.112875 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-e571fb2e-b805-49aa-bab7-2e78868a4e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:24.198816 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d78107-0357-4283-b0a3-a8f74535ae3e None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:24.204057 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 206/399] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 16:02:24.245594 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e571fb2e-b805-49aa-bab7-2e78868a4e1d None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:24.263074 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2065e23b-8b52-4d10-a7fc-a1887725f48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:24.263836 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065e23b-8b52-4d10-a7fc-a1887725f48f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:24.264119 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065e23b-8b52-4d10-a7fc-a1887725f48f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:24.264392 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065e23b-8b52-4d10-a7fc-a1887725f48f None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:24.266412 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2065e23b-8b52-4d10-a7fc-a1887725f48f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:24.266544 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2065e23b-8b52-4d10-a7fc-a1887725f48f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:24.267992 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2065e23b-8b52-4d10-a7fc-a1887725f48f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:24.279352 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 194/400] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:02:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:02:24.284866 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2065e23b-8b52-4d10-a7fc-a1887725f48f None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:24.312191 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:24.374296 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:24.408496 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e571fb2e-b805-49aa-bab7-2e78868a4e1d None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:24.425697 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2065e23b-8b52-4d10-a7fc-a1887725f48f None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:24.457747 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f1be4c-237b-4a38-8721-c620a0c18b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:24.458867 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f1be4c-237b-4a38-8721-c620a0c18b4d None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:24.459320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f1be4c-237b-4a38-8721-c620a0c18b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:24.459741 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f1be4c-237b-4a38-8721-c620a0c18b4d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:24.461481 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762df50d-71e6-4e32-94c9-3d456bb150c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:24.462409 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762df50d-71e6-4e32-94c9-3d456bb150c6 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:24.462761 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762df50d-71e6-4e32-94c9-3d456bb150c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:24.463102 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762df50d-71e6-4e32-94c9-3d456bb150c6 None admin] PATH_INFO: `/v3/groups/ad54dac2940a42e194222806ee8b60e9` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:24.464399 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762df50d-71e6-4e32-94c9-3d456bb150c6 None admin] RBAC: Authorizing `identity:get_group(group_id=ad54dac2940a42e194222806ee8b60e9)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:24.465908 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762df50d-71e6-4e32-94c9-3d456bb150c6 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:24.467816 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 195/401] 10.222.0.41 () {56 vars in 1180 bytes} [Fri Jun 3 16:02:24 2022] GET /identity/v3/groups/ad54dac2940a42e194222806ee8b60e9 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:02:24.469060 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ae750-bfdf-482a-8cff-794cadb78192 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:24.469445 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ae750-bfdf-482a-8cff-794cadb78192 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:24.469817 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ae750-bfdf-482a-8cff-794cadb78192 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:24.471376 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 196/402] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:24.473496 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f1be4c-237b-4a38-8721-c620a0c18b4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:24.476137 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ae750-bfdf-482a-8cff-794cadb78192 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:24.483099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-f1f1be4c-237b-4a38-8721-c620a0c18b4d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 16:02:24.483099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:24.483099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:24.483099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:24.483099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:24.483099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:24.483099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:24.483099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:24.483099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:24.483099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:24.483099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:24.483099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:24.483099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:24.483099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:24.483099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:24.483099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:24.484208 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:24.484208 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:24.484208 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:24.484208 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:24.484208 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:24.484208 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:24.484208 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:24.484208 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:24.484208 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:24.484208 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:24.484208 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:24.484208 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:24.484208 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:24.484208 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:24.484208 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:24.484208 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:24.485166 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:24.485166 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:24.485166 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:24.485166 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:24.485166 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:24.485166 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:24.485166 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:24.485166 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:24.485166 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:24.485166 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:24.485166 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 16:02:24.485166 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:24.492401 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 207/403] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:02:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:02:24.493944 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1bfe32-aeb7-4aca-81e0-76c1a14cbe4f None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:24.494408 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1bfe32-aeb7-4aca-81e0-76c1a14cbe4f None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:24.494831 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1bfe32-aeb7-4aca-81e0-76c1a14cbe4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:24.513068 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-4e1bfe32-aeb7-4aca-81e0-76c1a14cbe4f None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:24.514443 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-4e1bfe32-aeb7-4aca-81e0-76c1a14cbe4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:24.568088 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:24.586247 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 208/404] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:24.587524 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1bfe32-aeb7-4aca-81e0-76c1a14cbe4f None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:24.595458 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ae750-bfdf-482a-8cff-794cadb78192 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:24.635531 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ba5ee-cd69-4401-93c8-607c37434663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:24.636506 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ba5ee-cd69-4401-93c8-607c37434663 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:24.636855 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ba5ee-cd69-4401-93c8-607c37434663 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:24.637198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ba5ee-cd69-4401-93c8-607c37434663 None admin] PATH_INFO: `/v3/projects/23d03cc631264a3faa207c1b3bded8e3` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:24.638456 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85ba5ee-cd69-4401-93c8-607c37434663 None admin] RBAC: Authorizing `identity:get_project(project_id=23d03cc631264a3faa207c1b3bded8e3)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:24.640005 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85ba5ee-cd69-4401-93c8-607c37434663 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:24.642197 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 197/405] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:02:24 2022] GET /identity/v3/projects/23d03cc631264a3faa207c1b3bded8e3 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:24.643048 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ba5ee-cd69-4401-93c8-607c37434663 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:24.660499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:24.705429 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1bfe32-aeb7-4aca-81e0-76c1a14cbe4f None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:24.746753 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ba5ee-cd69-4401-93c8-607c37434663 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:24.751790 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76e836-2711-4395-97e2-1be4a8e1b3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:24.752699 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76e836-2711-4395-97e2-1be4a8e1b3c9 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:24.753208 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76e836-2711-4395-97e2-1be4a8e1b3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:24.753654 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76e836-2711-4395-97e2-1be4a8e1b3c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:24.755720 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76e836-2711-4395-97e2-1be4a8e1b3c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:24.756036 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76e836-2711-4395-97e2-1be4a8e1b3c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:24.757405 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76e836-2711-4395-97e2-1be4a8e1b3c9 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:24.774483 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 209/406] 10.222.0.41 () {56 vars in 1101 bytes} [Fri Jun 3 16:02:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:24.776003 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76e836-2711-4395-97e2-1be4a8e1b3c9 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:24.789168 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a5832-eb63-4144-bd2f-afb4e50bf474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:24.789950 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a5832-eb63-4144-bd2f-afb4e50bf474 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:24.790675 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a5832-eb63-4144-bd2f-afb4e50bf474 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:24.791089 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a5832-eb63-4144-bd2f-afb4e50bf474 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:24.801521 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4a5832-eb63-4144-bd2f-afb4e50bf474 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:24.809029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-fa4a5832-eb63-4144-bd2f-afb4e50bf474 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:02:24.809029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:24.809029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:24.809029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:24.809029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:24.809029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:24.809029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:24.809029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:24.809029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:24.809029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:24.809029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:24.809029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:24.809029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:24.809029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:24.809029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:24.809029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:24.809911 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:24.809911 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:24.809911 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:24.809911 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:24.809911 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:24.809911 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:24.809911 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:24.809911 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:24.809911 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:24.809911 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:24.809911 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:24.809911 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:24.809911 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:24.809911 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:24.809911 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:24.809911 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:24.810698 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:24.810698 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:24.810698 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:24.810698 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:24.810698 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:24.810698 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:24.810698 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:24.810698 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:24.810698 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:24.810698 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:24.810698 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:02:24.810698 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:24.814990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 198/407] 10.222.0.41 () {56 vars in 1096 bytes} [Fri Jun 3 16:02:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:24.815728 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a5832-eb63-4144-bd2f-afb4e50bf474 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:24.827836 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:24.902013 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76e836-2711-4395-97e2-1be4a8e1b3c9 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:24.932941 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a5832-eb63-4144-bd2f-afb4e50bf474 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:24.943077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:24.953001 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b90458-4746-4417-b760-1b192b196d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:24.954364 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b90458-4746-4417-b760-1b192b196d1e None admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:24.954863 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b90458-4746-4417-b760-1b192b196d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:24.956945 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b90458-4746-4417-b760-1b192b196d1e None admin] PATH_INFO: `/v3/projects/23d03cc631264a3faa207c1b3bded8e3/groups/ad54dac2940a42e194222806ee8b60e9/roles/2248ac4d70c34e459b8942dec0d7656b` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:24.962146 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b90458-4746-4417-b760-1b192b196d1e None admin] RBAC: Authorizing `identity:create_grant(project_id=23d03cc631264a3faa207c1b3bded8e3, group_id=ad54dac2940a42e194222806ee8b60e9, role_id=2248ac4d70c34e459b8942dec0d7656b)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:24.963653 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b90458-4746-4417-b760-1b192b196d1e None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:24.978376 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce496d-d31a-4d14-9154-74fdca0e037a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:24.979436 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 210/408] 10.222.0.41 () {58 vars in 1442 bytes} [Fri Jun 3 16:02:24 2022] PUT /identity/v3/projects/23d03cc631264a3faa207c1b3bded8e3/groups/ad54dac2940a42e194222806ee8b60e9/roles/2248ac4d70c34e459b8942dec0d7656b => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:02:24.979923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce496d-d31a-4d14-9154-74fdca0e037a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:24.980417 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b90458-4746-4417-b760-1b192b196d1e None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:24.980685 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce496d-d31a-4d14-9154-74fdca0e037a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:24.981107 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce496d-d31a-4d14-9154-74fdca0e037a None admin] PATH_INFO: `/v3/groups/ad54dac2940a42e194222806ee8b60e9` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:24.982684 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cce496d-d31a-4d14-9154-74fdca0e037a None admin] RBAC: Authorizing `identity:get_group(group_id=ad54dac2940a42e194222806ee8b60e9)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:24.985933 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cce496d-d31a-4d14-9154-74fdca0e037a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:24.988518 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 199/409] 10.222.0.41 () {56 vars in 1180 bytes} [Fri Jun 3 16:02:24 2022] GET /identity/v3/groups/ad54dac2940a42e194222806ee8b60e9 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:24.998701 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce496d-d31a-4d14-9154-74fdca0e037a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:25.099829 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:25.131242 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce496d-d31a-4d14-9154-74fdca0e037a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:25.175605 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6920a646-1d24-4951-8a4b-b31db42c55bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:25.176962 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b90458-4746-4417-b760-1b192b196d1e None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:25.177074 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920a646-1d24-4951-8a4b-b31db42c55bd None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:25.177278 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920a646-1d24-4951-8a4b-b31db42c55bd None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:25.177555 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920a646-1d24-4951-8a4b-b31db42c55bd None admin] PATH_INFO: `/v3/projects/23d03cc631264a3faa207c1b3bded8e3` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:25.178912 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6920a646-1d24-4951-8a4b-b31db42c55bd None admin] RBAC: Authorizing `identity:get_project(project_id=23d03cc631264a3faa207c1b3bded8e3)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:25.180426 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6920a646-1d24-4951-8a4b-b31db42c55bd None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:25.182505 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 200/410] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:02:24 2022] GET /identity/v3/projects/23d03cc631264a3faa207c1b3bded8e3 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:25.189656 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6920a646-1d24-4951-8a4b-b31db42c55bd None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:25.235211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8962b-7330-4fdf-8609-ebfc7ddd13fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:25.235965 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8962b-7330-4fdf-8609-ebfc7ddd13fb None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:25.236230 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8962b-7330-4fdf-8609-ebfc7ddd13fb None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:25.236510 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8962b-7330-4fdf-8609-ebfc7ddd13fb None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:25.237263 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d8962b-7330-4fdf-8609-ebfc7ddd13fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:25.237573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d8962b-7330-4fdf-8609-ebfc7ddd13fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:25.240048 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d8962b-7330-4fdf-8609-ebfc7ddd13fb None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:25.255469 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 211/411] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:02:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:25.260433 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8962b-7330-4fdf-8609-ebfc7ddd13fb None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:25.330402 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:25.332091 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6920a646-1d24-4951-8a4b-b31db42c55bd None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:25.394759 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f68c0-a308-4f0d-8b29-60df199d7be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:25.395711 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f68c0-a308-4f0d-8b29-60df199d7be5 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:25.396000 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f68c0-a308-4f0d-8b29-60df199d7be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:25.396280 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f68c0-a308-4f0d-8b29-60df199d7be5 None admin] PATH_INFO: `/v3/projects/23d03cc631264a3faa207c1b3bded8e3/groups/ad54dac2940a42e194222806ee8b60e9/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:25.399453 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f68c0-a308-4f0d-8b29-60df199d7be5 None admin] RBAC: Authorizing `identity:create_grant(project_id=23d03cc631264a3faa207c1b3bded8e3, group_id=ad54dac2940a42e194222806ee8b60e9, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:25.400909 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f68c0-a308-4f0d-8b29-60df199d7be5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:25.415733 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8962b-7330-4fdf-8609-ebfc7ddd13fb None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:25.425501 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 201/412] 10.222.0.41 () {58 vars in 1442 bytes} [Fri Jun 3 16:02:25 2022] PUT /identity/v3/projects/23d03cc631264a3faa207c1b3bded8e3/groups/ad54dac2940a42e194222806ee8b60e9/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:02:25.480176 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751baaf3-4598-471f-b8cb-e5fde21487fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:25.481073 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751baaf3-4598-471f-b8cb-e5fde21487fb None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:25.481362 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751baaf3-4598-471f-b8cb-e5fde21487fb None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:25.481642 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751baaf3-4598-471f-b8cb-e5fde21487fb None admin] PATH_INFO: `/v3/groups/06ee27f162d145e79244190e976f5e0d` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:25.488191 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751baaf3-4598-471f-b8cb-e5fde21487fb None admin] RBAC: Authorizing `identity:get_group(group_id=06ee27f162d145e79244190e976f5e0d)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:25.490393 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751baaf3-4598-471f-b8cb-e5fde21487fb None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:25.492604 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 212/413] 10.222.0.41 () {56 vars in 1180 bytes} [Fri Jun 3 16:02:25 2022] GET /identity/v3/groups/06ee27f162d145e79244190e976f5e0d => generated 229 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:25.500515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f68c0-a308-4f0d-8b29-60df199d7be5 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:25.504037 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:25.658391 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f68c0-a308-4f0d-8b29-60df199d7be5 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:25.697379 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0a7de-2b63-4c40-b1a5-c9524d3e763b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:25.698250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0a7de-2b63-4c40-b1a5-c9524d3e763b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:25.698560 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0a7de-2b63-4c40-b1a5-c9524d3e763b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:25.698807 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0a7de-2b63-4c40-b1a5-c9524d3e763b None admin] PATH_INFO: `/v3/projects/450b64d0f10f4cb8949a26d9eea4aef4` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:25.700027 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0a7de-2b63-4c40-b1a5-c9524d3e763b None admin] RBAC: Authorizing `identity:get_project(project_id=450b64d0f10f4cb8949a26d9eea4aef4)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:25.701560 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0a7de-2b63-4c40-b1a5-c9524d3e763b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:25.703584 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 202/414] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:02:25 2022] GET /identity/v3/projects/450b64d0f10f4cb8949a26d9eea4aef4 => generated 344 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:25.709727 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751baaf3-4598-471f-b8cb-e5fde21487fb None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:25.864562 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751baaf3-4598-471f-b8cb-e5fde21487fb None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:25.888959 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:25.910393 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7ad41-7692-4104-98a8-d85a8c8f448e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:25.911807 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7ad41-7692-4104-98a8-d85a8c8f448e None admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:25.912100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7ad41-7692-4104-98a8-d85a8c8f448e None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:25.912382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7ad41-7692-4104-98a8-d85a8c8f448e None admin] PATH_INFO: `/v3/projects/450b64d0f10f4cb8949a26d9eea4aef4/groups/06ee27f162d145e79244190e976f5e0d/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:25.920055 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d7ad41-7692-4104-98a8-d85a8c8f448e None admin] RBAC: Authorizing `identity:create_grant(project_id=450b64d0f10f4cb8949a26d9eea4aef4, group_id=06ee27f162d145e79244190e976f5e0d, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:25.921498 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d7ad41-7692-4104-98a8-d85a8c8f448e None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:25.950581 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 213/415] 10.222.0.41 () {58 vars in 1442 bytes} [Fri Jun 3 16:02:25 2022] PUT /identity/v3/projects/450b64d0f10f4cb8949a26d9eea4aef4/groups/06ee27f162d145e79244190e976f5e0d/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:02:26.298707 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4862e-6938-45e9-abb0-d49bde5f358d None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:26.299053 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4862e-6938-45e9-abb0-d49bde5f358d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:26.299274 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4862e-6938-45e9-abb0-d49bde5f358d None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:26.301003 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 203/416] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:26.307391 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4a594-638b-40b4-bffc-cc913d38abc0 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:26.307909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4a594-638b-40b4-bffc-cc913d38abc0 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:26.308017 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4a594-638b-40b4-bffc-cc913d38abc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:26.326529 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-8da4a594-638b-40b4-bffc-cc913d38abc0 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:26.327812 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-8da4a594-638b-40b4-bffc-cc913d38abc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:26.407407 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 214/417] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:26.426003 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e4862e-6938-45e9-abb0-d49bde5f358d None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:26.506780 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:26.559995 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e4862e-6938-45e9-abb0-d49bde5f358d None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:26.602338 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e80fbe8-dbae-41f3-aa3f-b0d2df0b663c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:26.603210 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e80fbe8-dbae-41f3-aa3f-b0d2df0b663c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:26.603533 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e80fbe8-dbae-41f3-aa3f-b0d2df0b663c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:26.603818 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e80fbe8-dbae-41f3-aa3f-b0d2df0b663c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:26.619116 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e80fbe8-dbae-41f3-aa3f-b0d2df0b663c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:26.627333 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-1e80fbe8-dbae-41f3-aa3f-b0d2df0b663c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 16:02:26.627333 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:26.627333 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:26.627333 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:26.627333 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:26.627333 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:26.627333 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:26.627333 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:26.627333 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:26.627333 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:26.627333 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:26.627333 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:26.627333 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:26.627333 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:26.627333 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:26.627333 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:26.628206 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:26.628206 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:26.628206 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:26.628206 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:26.628206 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:26.628206 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:26.628206 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:26.628206 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:26.628206 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:26.628206 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:26.628206 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:26.628206 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:26.628206 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:26.628206 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:26.628206 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:26.628206 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:26.629006 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:26.629006 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:26.629006 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:26.629006 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:26.629006 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:26.629006 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:26.629006 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:26.629006 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:26.629006 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:26.629006 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:26.629006 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 16:02:26.629006 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:26.629006 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 204/418] 10.222.0.41 () {56 vars in 1114 bytes} [Fri Jun 3 16:02:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:26.636411 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4a594-638b-40b4-bffc-cc913d38abc0 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:26.682103 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db8bbb-1251-4b76-ae9f-52c2ca1dfcad None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:26.682731 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db8bbb-1251-4b76-ae9f-52c2ca1dfcad None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:26.683208 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db8bbb-1251-4b76-ae9f-52c2ca1dfcad None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:26.685083 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 205/419] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:26.690384 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d0bad-57cb-47b6-83dc-66e543b47647 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:26.691157 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d0bad-57cb-47b6-83dc-66e543b47647 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:26.691764 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d0bad-57cb-47b6-83dc-66e543b47647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:26.709897 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-167d0bad-57cb-47b6-83dc-66e543b47647 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:26.711467 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-167d0bad-57cb-47b6-83dc-66e543b47647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:26.822554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 206/420] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 16:02:26.828418 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4a594-638b-40b4-bffc-cc913d38abc0 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:26.841763 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:26.861189 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d0bad-57cb-47b6-83dc-66e543b47647 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:26.883988 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a82aa4c-53ce-4c6b-ba18-44976c37f120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:26.884665 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a82aa4c-53ce-4c6b-ba18-44976c37f120 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:26.884951 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a82aa4c-53ce-4c6b-ba18-44976c37f120 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:26.885226 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a82aa4c-53ce-4c6b-ba18-44976c37f120 None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:26.885964 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a82aa4c-53ce-4c6b-ba18-44976c37f120 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:26.886293 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a82aa4c-53ce-4c6b-ba18-44976c37f120 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:26.887681 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a82aa4c-53ce-4c6b-ba18-44976c37f120 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:26.906949 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 215/421] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:02:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:26.912208 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a82aa4c-53ce-4c6b-ba18-44976c37f120 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:26.947655 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:27.011378 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d0bad-57cb-47b6-83dc-66e543b47647 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:27.074967 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77668118-f648-4e07-94e4-c83cc99ae2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:27.075704 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77668118-f648-4e07-94e4-c83cc99ae2b8 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:27.075990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77668118-f648-4e07-94e4-c83cc99ae2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:27.076267 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77668118-f648-4e07-94e4-c83cc99ae2b8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:27.086217 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a82aa4c-53ce-4c6b-ba18-44976c37f120 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:27.099756 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77668118-f648-4e07-94e4-c83cc99ae2b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:27.112250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-77668118-f648-4e07-94e4-c83cc99ae2b8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 16:02:27.112250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:27.112250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:27.112250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:27.112250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:27.112250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:27.112250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:27.112250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:27.112250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:27.112250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:27.112250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:27.112250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:27.112250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:27.112250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:27.112250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:27.112250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:27.113167 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:27.113167 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:27.113167 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:27.113167 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:27.113167 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:27.113167 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:27.113167 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:27.113167 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:27.113167 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:27.113167 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:27.113167 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:27.113167 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:27.113167 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:27.113167 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:27.113167 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:27.113167 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:27.114192 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:27.114192 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:27.114192 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:27.114192 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:27.114192 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:27.114192 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:27.114192 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:27.114192 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:27.114192 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:27.114192 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:27.114192 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 16:02:27.114192 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:27.115181 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 207/422] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:02:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:02:27.123012 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77668118-f648-4e07-94e4-c83cc99ae2b8 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:27.144699 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4523ca-23e3-4356-b747-55212e86a017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:27.146302 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4523ca-23e3-4356-b747-55212e86a017 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:27.146787 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4523ca-23e3-4356-b747-55212e86a017 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:27.147233 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4523ca-23e3-4356-b747-55212e86a017 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:27.228010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:27.259389 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77668118-f648-4e07-94e4-c83cc99ae2b8 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:27.278459 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4523ca-23e3-4356-b747-55212e86a017 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:27.280200 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4523ca-23e3-4356-b747-55212e86a017 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:27.301294 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:27.326261 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82551e01-6362-4b9e-b2d3-00322393c849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:27.326854 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82551e01-6362-4b9e-b2d3-00322393c849 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:27.327135 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82551e01-6362-4b9e-b2d3-00322393c849 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:27.327434 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82551e01-6362-4b9e-b2d3-00322393c849 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:27.328191 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82551e01-6362-4b9e-b2d3-00322393c849 None admin] RBAC: Adding query filter params (name=member) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:27.328495 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82551e01-6362-4b9e-b2d3-00322393c849 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:27.329872 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82551e01-6362-4b9e-b2d3-00322393c849 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:27.349708 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 208/423] 10.222.0.41 () {56 vars in 1101 bytes} [Fri Jun 3 16:02:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:27.356766 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82551e01-6362-4b9e-b2d3-00322393c849 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:27.462110 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 216/424] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:02:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 03 16:02:27.467644 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4523ca-23e3-4356-b747-55212e86a017 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:27.555986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:27.559696 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82551e01-6362-4b9e-b2d3-00322393c849 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:27.590349 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4523ca-23e3-4356-b747-55212e86a017 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:27.627828 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d1e58-9274-46bb-9518-8949e4b052ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:27.628701 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d1e58-9274-46bb-9518-8949e4b052ac None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:27.628984 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d1e58-9274-46bb-9518-8949e4b052ac None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:27.629295 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d1e58-9274-46bb-9518-8949e4b052ac None admin] PATH_INFO: `/v3/projects/1d2895687ae2471682919de9b8fa4afb` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:27.630501 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5d1e58-9274-46bb-9518-8949e4b052ac None admin] RBAC: Authorizing `identity:get_project(project_id=1d2895687ae2471682919de9b8fa4afb)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:27.632349 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5d1e58-9274-46bb-9518-8949e4b052ac None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:27.634401 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 217/425] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:02:27 2022] GET /identity/v3/projects/1d2895687ae2471682919de9b8fa4afb => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:27.642786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d1e58-9274-46bb-9518-8949e4b052ac None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:27.662509 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e529753a-af80-41f7-8b75-d228f9d75324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:27.663169 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e529753a-af80-41f7-8b75-d228f9d75324 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:27.663482 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e529753a-af80-41f7-8b75-d228f9d75324 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:27.663761 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e529753a-af80-41f7-8b75-d228f9d75324 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:27.733537 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:27.755238 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e529753a-af80-41f7-8b75-d228f9d75324 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:27.756796 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e529753a-af80-41f7-8b75-d228f9d75324 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:27.870298 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 209/426] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:02:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 03 16:02:27.876135 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e529753a-af80-41f7-8b75-d228f9d75324 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:27.876628 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d1e58-9274-46bb-9518-8949e4b052ac None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:27.918925 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041da56-4106-4e9e-843d-21d79f6076cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:27.919821 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041da56-4106-4e9e-843d-21d79f6076cb None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:27.920108 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041da56-4106-4e9e-843d-21d79f6076cb None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:27.920386 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041da56-4106-4e9e-843d-21d79f6076cb None admin] PATH_INFO: `/v3/groups/ad54dac2940a42e194222806ee8b60e9` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:27.921568 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8041da56-4106-4e9e-843d-21d79f6076cb None admin] RBAC: Authorizing `identity:get_group(group_id=ad54dac2940a42e194222806ee8b60e9)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:27.922996 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8041da56-4106-4e9e-843d-21d79f6076cb None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:27.924945 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 218/427] 10.222.0.41 () {56 vars in 1180 bytes} [Fri Jun 3 16:02:27 2022] GET /identity/v3/groups/ad54dac2940a42e194222806ee8b60e9 => generated 226 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:02:27.926165 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbc0a7-c995-4057-b324-5d141fb1ffe0 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:27.926456 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbc0a7-c995-4057-b324-5d141fb1ffe0 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:27.926738 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbc0a7-c995-4057-b324-5d141fb1ffe0 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:27.928118 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 219/428] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:27.933201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe5354-aadc-4262-b68a-376970a9fbc6 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:27.933494 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe5354-aadc-4262-b68a-376970a9fbc6 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:27.933774 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe5354-aadc-4262-b68a-376970a9fbc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:27.950642 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-59fe5354-aadc-4262-b68a-376970a9fbc6 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:27.951868 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-59fe5354-aadc-4262-b68a-376970a9fbc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:27.972329 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:27.982649 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:28.006220 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e529753a-af80-41f7-8b75-d228f9d75324 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:28.031404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 220/429] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 16:02:28.032329 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe5354-aadc-4262-b68a-376970a9fbc6 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:28.045449 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f59a97c-826f-404a-a9ee-8836ba27752c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:28.046311 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59a97c-826f-404a-a9ee-8836ba27752c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:28.046605 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59a97c-826f-404a-a9ee-8836ba27752c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:28.046879 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59a97c-826f-404a-a9ee-8836ba27752c None admin] PATH_INFO: `/v3/projects/1d2895687ae2471682919de9b8fa4afb` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:28.048509 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f59a97c-826f-404a-a9ee-8836ba27752c None admin] RBAC: Authorizing `identity:get_project(project_id=1d2895687ae2471682919de9b8fa4afb)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:28.049967 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f59a97c-826f-404a-a9ee-8836ba27752c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:28.057332 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 210/430] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:02:27 2022] GET /identity/v3/projects/1d2895687ae2471682919de9b8fa4afb => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:02:28.058610 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f59a97c-826f-404a-a9ee-8836ba27752c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:28.156397 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe5354-aadc-4262-b68a-376970a9fbc6 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:28.188878 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f59a97c-826f-404a-a9ee-8836ba27752c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:28.197046 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b37e90a-471b-42c3-a397-14f41c193b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:28.197715 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37e90a-471b-42c3-a397-14f41c193b46 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:28.197999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37e90a-471b-42c3-a397-14f41c193b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:28.198281 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37e90a-471b-42c3-a397-14f41c193b46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:28.199145 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b37e90a-471b-42c3-a397-14f41c193b46 None admin] RBAC: Adding query filter params (group.id=ad54dac2940a42e194222806ee8b60e9, role.id=2248ac4d70c34e459b8942dec0d7656b, scope.project.id=1d2895687ae2471682919de9b8fa4afb) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:28.199513 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b37e90a-471b-42c3-a397-14f41c193b46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:28.201568 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b37e90a-471b-42c3-a397-14f41c193b46 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:28.210624 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 221/431] 10.222.0.41 () {56 vars in 1376 bytes} [Fri Jun 3 16:02:28 2022] GET /identity/v3/role_assignments?group.id=ad54dac2940a42e194222806ee8b60e9&scope.project.id=1d2895687ae2471682919de9b8fa4afb&role.id=2248ac4d70c34e459b8942dec0d7656b => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:28.211573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b37e90a-471b-42c3-a397-14f41c193b46 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:28.222484 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:28.237048 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426e03b-6550-4ebe-870f-c9596be6766e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:28.238237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426e03b-6550-4ebe-870f-c9596be6766e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:28.238739 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426e03b-6550-4ebe-870f-c9596be6766e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:28.239168 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426e03b-6550-4ebe-870f-c9596be6766e None admin] PATH_INFO: `/v3/groups/ad54dac2940a42e194222806ee8b60e9` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:28.240658 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2426e03b-6550-4ebe-870f-c9596be6766e None admin] RBAC: Authorizing `identity:get_group(group_id=ad54dac2940a42e194222806ee8b60e9)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:28.242319 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2426e03b-6550-4ebe-870f-c9596be6766e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:28.248836 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 211/432] 10.222.0.41 () {56 vars in 1180 bytes} [Fri Jun 3 16:02:28 2022] GET /identity/v3/groups/ad54dac2940a42e194222806ee8b60e9 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:02:28.248836 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86155e-a93a-43a6-b509-595413433927 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:28.248836 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86155e-a93a-43a6-b509-595413433927 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:28.248836 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86155e-a93a-43a6-b509-595413433927 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:28.248836 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 212/433] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:28.254689 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d86155e-a93a-43a6-b509-595413433927 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:28.280396 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:28.357308 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b37e90a-471b-42c3-a397-14f41c193b46 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:28.400415 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200fa230-bd10-4288-a117-cd95b7db89ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:28.401309 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fa230-bd10-4288-a117-cd95b7db89ee None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:28.401597 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fa230-bd10-4288-a117-cd95b7db89ee None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:28.401920 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fa230-bd10-4288-a117-cd95b7db89ee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:28.412504 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d86155e-a93a-43a6-b509-595413433927 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:28.414967 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200fa230-bd10-4288-a117-cd95b7db89ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:28.423320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-200fa230-bd10-4288-a117-cd95b7db89ee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 16:02:28.423320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:28.423320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:28.423320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:28.423320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:28.423320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:28.423320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:28.423320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:28.423320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:28.423320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:28.423320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:28.423320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:28.423320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:28.423320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:28.423320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:28.423320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:28.424224 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:28.424224 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:28.424224 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:28.424224 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:28.424224 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:28.424224 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:28.424224 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:28.424224 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:28.424224 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:28.424224 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:28.424224 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:28.424224 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:28.424224 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:28.424224 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:28.424224 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:28.424224 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:28.425035 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:28.425035 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:28.425035 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:28.425035 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:28.425035 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:28.425035 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:28.425035 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:28.425035 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:28.425035 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:28.425035 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:28.425035 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 16:02:28.425035 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:28.425035 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 222/434] 10.222.0.41 () {56 vars in 1114 bytes} [Fri Jun 3 16:02:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:28.426966 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0683c2-f82e-4ee3-ab48-79d0637fe54f None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:28.427258 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0683c2-f82e-4ee3-ab48-79d0637fe54f None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:28.427562 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0683c2-f82e-4ee3-ab48-79d0637fe54f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:28.447071 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-9a0683c2-f82e-4ee3-ab48-79d0637fe54f None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:28.448656 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-9a0683c2-f82e-4ee3-ab48-79d0637fe54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:28.456263 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e2d00-2748-4f64-83c4-a61a8fcca0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:28.457162 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e2d00-2748-4f64-83c4-a61a8fcca0ec None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:28.458417 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e2d00-2748-4f64-83c4-a61a8fcca0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:28.458417 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e2d00-2748-4f64-83c4-a61a8fcca0ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:28.459187 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e2d00-2748-4f64-83c4-a61a8fcca0ec None admin] RBAC: Adding query filter params (group.id=ad54dac2940a42e194222806ee8b60e9, role.id=a8e4174e3572497f9284c709a119dcff, scope.project.id=1d2895687ae2471682919de9b8fa4afb) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:28.459532 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e2d00-2748-4f64-83c4-a61a8fcca0ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:28.461802 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e2d00-2748-4f64-83c4-a61a8fcca0ec None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:28.469576 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 213/435] 10.222.0.41 () {56 vars in 1376 bytes} [Fri Jun 3 16:02:28 2022] GET /identity/v3/role_assignments?group.id=ad54dac2940a42e194222806ee8b60e9&scope.project.id=1d2895687ae2471682919de9b8fa4afb&role.id=a8e4174e3572497f9284c709a119dcff => generated 611 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:28.470702 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e2d00-2748-4f64-83c4-a61a8fcca0ec None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:28.529448 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:28.543752 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 223/436] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:28.568381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0683c2-f82e-4ee3-ab48-79d0637fe54f None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:28.591462 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:28.611721 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e2d00-2748-4f64-83c4-a61a8fcca0ec None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:28.658314 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b572a93-1721-493c-b54e-ac0c9e5d8985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:28.658994 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b572a93-1721-493c-b54e-ac0c9e5d8985 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:28.659276 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b572a93-1721-493c-b54e-ac0c9e5d8985 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:28.659732 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b572a93-1721-493c-b54e-ac0c9e5d8985 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:28.660511 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b572a93-1721-493c-b54e-ac0c9e5d8985 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:28.661167 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b572a93-1721-493c-b54e-ac0c9e5d8985 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:28.662388 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b572a93-1721-493c-b54e-ac0c9e5d8985 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:28.675952 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 214/437] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:02:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:28.678330 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b572a93-1721-493c-b54e-ac0c9e5d8985 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:28.716792 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0683c2-f82e-4ee3-ab48-79d0637fe54f None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:28.763960 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850c4c1-f805-4dda-80a4-b2d3045c60e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:28.765383 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850c4c1-f805-4dda-80a4-b2d3045c60e5 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:28.766035 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850c4c1-f805-4dda-80a4-b2d3045c60e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:28.767286 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850c4c1-f805-4dda-80a4-b2d3045c60e5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:28.782469 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a850c4c1-f805-4dda-80a4-b2d3045c60e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:28.794382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-a850c4c1-f805-4dda-80a4-b2d3045c60e5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 16:02:28.794382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:28.794382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:28.794382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:28.794382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:28.794382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:28.794382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:28.794382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:28.794382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:28.794382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:28.794382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:28.794382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:28.794382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:28.794382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:28.794382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:28.794382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:28.797075 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:28.797075 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:28.797075 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:28.797075 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:28.797075 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:28.797075 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:28.797075 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:28.797075 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:28.797075 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:28.797075 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:28.797075 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:28.797075 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:28.797075 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:28.797075 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:28.797075 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:28.797075 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:28.797926 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:28.797926 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:28.797926 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:28.797926 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:28.797926 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:28.797926 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:28.797926 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:28.797926 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:28.797926 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:28.797926 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:28.797926 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 16:02:28.797926 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:28.797926 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 224/438] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:02:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:02:28.803686 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850c4c1-f805-4dda-80a4-b2d3045c60e5 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:28.807071 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b572a93-1721-493c-b54e-ac0c9e5d8985 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:28.849918 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a6d50-138d-47f8-ae20-98fb9003e088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:28.850565 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a6d50-138d-47f8-ae20-98fb9003e088 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:28.850851 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a6d50-138d-47f8-ae20-98fb9003e088 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:28.851141 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a6d50-138d-47f8-ae20-98fb9003e088 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:28.867591 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:28.878436 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:28.933327 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a6d50-138d-47f8-ae20-98fb9003e088 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:28.934722 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a6d50-138d-47f8-ae20-98fb9003e088 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:28.942008 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850c4c1-f805-4dda-80a4-b2d3045c60e5 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:28.987166 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ab26d-38c2-4317-93fc-4afec30e2081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:28.987996 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ab26d-38c2-4317-93fc-4afec30e2081 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:28.988281 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ab26d-38c2-4317-93fc-4afec30e2081 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:28.989083 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ab26d-38c2-4317-93fc-4afec30e2081 None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:28.989874 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ab26d-38c2-4317-93fc-4afec30e2081 None admin] RBAC: Adding query filter params (name=member) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:28.990190 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ab26d-38c2-4317-93fc-4afec30e2081 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:28.991649 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ab26d-38c2-4317-93fc-4afec30e2081 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:29.001502 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 225/439] 10.222.0.41 () {56 vars in 1101 bytes} [Fri Jun 3 16:02:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:29.002919 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d080e3b-fa90-462e-8742-f7cb30639853 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:29.003279 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d080e3b-fa90-462e-8742-f7cb30639853 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:29.003659 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d080e3b-fa90-462e-8742-f7cb30639853 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:29.005148 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 226/440] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:29.008968 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d080e3b-fa90-462e-8742-f7cb30639853 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:29.052318 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 215/441] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:02:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 03 16:02:29.053808 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364bec1-6f5a-47df-9bac-25f8e6c136e5 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:29.053996 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364bec1-6f5a-47df-9bac-25f8e6c136e5 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:29.054243 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364bec1-6f5a-47df-9bac-25f8e6c136e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:29.071874 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-d364bec1-6f5a-47df-9bac-25f8e6c136e5 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:29.073063 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d364bec1-6f5a-47df-9bac-25f8e6c136e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:29.119152 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:29.134779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d080e3b-fa90-462e-8742-f7cb30639853 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:29.146465 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 216/442] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:29.147317 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364bec1-6f5a-47df-9bac-25f8e6c136e5 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:29.150008 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:29.174473 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a82451-0200-4256-aa75-3e7bbd9741cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:29.175131 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a82451-0200-4256-aa75-3e7bbd9741cc None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:29.175451 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a82451-0200-4256-aa75-3e7bbd9741cc None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:29.175754 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a82451-0200-4256-aa75-3e7bbd9741cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:29.241106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a82451-0200-4256-aa75-3e7bbd9741cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:29.242463 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a82451-0200-4256-aa75-3e7bbd9741cc None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:29.265152 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364bec1-6f5a-47df-9bac-25f8e6c136e5 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:29.297782 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696abdf8-f259-4dc2-9370-95bb2e25818b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:29.298632 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696abdf8-f259-4dc2-9370-95bb2e25818b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:29.298913 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696abdf8-f259-4dc2-9370-95bb2e25818b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:29.299185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696abdf8-f259-4dc2-9370-95bb2e25818b None admin] PATH_INFO: `/v3/projects/23d03cc631264a3faa207c1b3bded8e3` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:29.300384 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696abdf8-f259-4dc2-9370-95bb2e25818b None admin] RBAC: Authorizing `identity:get_project(project_id=23d03cc631264a3faa207c1b3bded8e3)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:29.301855 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696abdf8-f259-4dc2-9370-95bb2e25818b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:29.303546 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 217/443] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:02:29 2022] GET /identity/v3/projects/23d03cc631264a3faa207c1b3bded8e3 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:29.304297 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696abdf8-f259-4dc2-9370-95bb2e25818b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:29.340464 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 227/444] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:02:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 03 16:02:29.341294 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a82451-0200-4256-aa75-3e7bbd9741cc None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:29.349840 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:29.356612 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:29.405660 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696abdf8-f259-4dc2-9370-95bb2e25818b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:29.438047 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1722f9-a38a-4321-b5b7-1f0f11a73e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:29.438823 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1722f9-a38a-4321-b5b7-1f0f11a73e0a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:29.439111 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1722f9-a38a-4321-b5b7-1f0f11a73e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:29.439315 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a82451-0200-4256-aa75-3e7bbd9741cc None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:29.439429 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1722f9-a38a-4321-b5b7-1f0f11a73e0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:29.449761 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1722f9-a38a-4321-b5b7-1f0f11a73e0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:29.457964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-0a1722f9-a38a-4321-b5b7-1f0f11a73e0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:02:29.457964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:29.457964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:29.457964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:29.457964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:29.457964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:29.457964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:29.457964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:29.457964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:29.457964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:29.457964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:29.457964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:29.457964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:29.457964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:29.457964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:29.457964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:29.458854 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:29.458854 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:29.458854 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:29.458854 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:29.458854 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:29.458854 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:29.458854 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:29.458854 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:29.458854 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:29.458854 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:29.458854 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:29.458854 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:29.458854 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:29.458854 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:29.458854 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:29.458854 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:29.464487 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:29.464487 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:29.464487 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:29.464487 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:29.464487 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:29.464487 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:29.464487 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:29.464487 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:29.464487 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:29.464487 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:29.464487 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:02:29.464487 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:29.464487 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 218/445] 10.222.0.41 () {56 vars in 1096 bytes} [Fri Jun 3 16:02:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:29.464487 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1722f9-a38a-4321-b5b7-1f0f11a73e0a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:29.472543 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8d9df-3562-4368-8221-702487cb20ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:29.473428 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8d9df-3562-4368-8221-702487cb20ee None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:29.473708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8d9df-3562-4368-8221-702487cb20ee None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:29.473980 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8d9df-3562-4368-8221-702487cb20ee None admin] PATH_INFO: `/v3/groups/ad54dac2940a42e194222806ee8b60e9` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:29.475123 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df8d9df-3562-4368-8221-702487cb20ee None admin] RBAC: Authorizing `identity:get_group(group_id=ad54dac2940a42e194222806ee8b60e9)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:29.476582 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df8d9df-3562-4368-8221-702487cb20ee None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:29.478170 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 228/446] 10.222.0.41 () {56 vars in 1180 bytes} [Fri Jun 3 16:02:29 2022] GET /identity/v3/groups/ad54dac2940a42e194222806ee8b60e9 => generated 226 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:29.480033 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8d9df-3562-4368-8221-702487cb20ee None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:29.560039 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1722f9-a38a-4321-b5b7-1f0f11a73e0a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:29.561462 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:29.565344 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:29.585519 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8d9df-3562-4368-8221-702487cb20ee None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:29.594164 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f0f94e-1052-4ce8-b248-b756fc3b0a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:29.595016 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0f94e-1052-4ce8-b248-b756fc3b0a8e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:29.595316 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0f94e-1052-4ce8-b248-b756fc3b0a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:29.595597 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0f94e-1052-4ce8-b248-b756fc3b0a8e None admin] PATH_INFO: `/v3/projects/23d03cc631264a3faa207c1b3bded8e3` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:29.596784 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f0f94e-1052-4ce8-b248-b756fc3b0a8e None admin] RBAC: Authorizing `identity:get_project(project_id=23d03cc631264a3faa207c1b3bded8e3)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:29.598222 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f0f94e-1052-4ce8-b248-b756fc3b0a8e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:29.599934 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 219/447] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:02:29 2022] GET /identity/v3/projects/23d03cc631264a3faa207c1b3bded8e3 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:29.600689 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f0f94e-1052-4ce8-b248-b756fc3b0a8e None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:29.619064 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f36227-6d1e-4301-b476-13f5eec2b950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:29.619748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f36227-6d1e-4301-b476-13f5eec2b950 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:29.620027 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f36227-6d1e-4301-b476-13f5eec2b950 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:29.620299 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f36227-6d1e-4301-b476-13f5eec2b950 None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:29.621027 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f36227-6d1e-4301-b476-13f5eec2b950 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:29.621330 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f36227-6d1e-4301-b476-13f5eec2b950 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:29.622666 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f36227-6d1e-4301-b476-13f5eec2b950 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:29.631039 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 229/448] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:02:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:29.631861 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f36227-6d1e-4301-b476-13f5eec2b950 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:29.697549 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f0f94e-1052-4ce8-b248-b756fc3b0a8e None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:29.730827 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5f4e5d-1075-473e-acff-f8e5d6d7498e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:29.731492 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5f4e5d-1075-473e-acff-f8e5d6d7498e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:29.731747 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f36227-6d1e-4301-b476-13f5eec2b950 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:29.731852 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5f4e5d-1075-473e-acff-f8e5d6d7498e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:29.732040 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5f4e5d-1075-473e-acff-f8e5d6d7498e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:29.732860 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5f4e5d-1075-473e-acff-f8e5d6d7498e None admin] RBAC: Adding query filter params (group.id=ad54dac2940a42e194222806ee8b60e9, role.id=2248ac4d70c34e459b8942dec0d7656b, scope.project.id=23d03cc631264a3faa207c1b3bded8e3) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:29.733173 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5f4e5d-1075-473e-acff-f8e5d6d7498e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:29.734602 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5f4e5d-1075-473e-acff-f8e5d6d7498e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:29.740648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 220/449] 10.222.0.41 () {56 vars in 1376 bytes} [Fri Jun 3 16:02:29 2022] GET /identity/v3/role_assignments?group.id=ad54dac2940a42e194222806ee8b60e9&scope.project.id=23d03cc631264a3faa207c1b3bded8e3&role.id=2248ac4d70c34e459b8942dec0d7656b => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:29.741334 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5f4e5d-1075-473e-acff-f8e5d6d7498e None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:29.779510 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40390698-249b-4305-8378-b03e8e158f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:29.780405 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40390698-249b-4305-8378-b03e8e158f44 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:29.780697 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40390698-249b-4305-8378-b03e8e158f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:29.780978 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40390698-249b-4305-8378-b03e8e158f44 None admin] PATH_INFO: `/v3/groups/ad54dac2940a42e194222806ee8b60e9` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:29.781634 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:29.782253 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40390698-249b-4305-8378-b03e8e158f44 None admin] RBAC: Authorizing `identity:get_group(group_id=ad54dac2940a42e194222806ee8b60e9)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:29.783777 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40390698-249b-4305-8378-b03e8e158f44 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:29.786231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 230/450] 10.222.0.41 () {56 vars in 1180 bytes} [Fri Jun 3 16:02:29 2022] GET /identity/v3/groups/ad54dac2940a42e194222806ee8b60e9 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:29.792781 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40390698-249b-4305-8378-b03e8e158f44 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:29.811579 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:29.861781 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5f4e5d-1075-473e-acff-f8e5d6d7498e None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:29.899750 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7456044c-c4c9-40e5-a676-772c6ee8c835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:29.900399 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456044c-c4c9-40e5-a676-772c6ee8c835 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:29.900683 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456044c-c4c9-40e5-a676-772c6ee8c835 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:29.900957 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456044c-c4c9-40e5-a676-772c6ee8c835 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:29.909252 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40390698-249b-4305-8378-b03e8e158f44 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:29.949725 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f860dd3-0662-4c3c-bc6f-e52410279e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:29.950651 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f860dd3-0662-4c3c-bc6f-e52410279e01 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:29.950931 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f860dd3-0662-4c3c-bc6f-e52410279e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:29.951403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f860dd3-0662-4c3c-bc6f-e52410279e01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:29.952523 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f860dd3-0662-4c3c-bc6f-e52410279e01 None admin] RBAC: Adding query filter params (group.id=ad54dac2940a42e194222806ee8b60e9, role.id=a8e4174e3572497f9284c709a119dcff, scope.project.id=23d03cc631264a3faa207c1b3bded8e3) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:29.952847 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f860dd3-0662-4c3c-bc6f-e52410279e01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:29.954283 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f860dd3-0662-4c3c-bc6f-e52410279e01 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:29.962700 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 231/451] 10.222.0.41 () {56 vars in 1376 bytes} [Fri Jun 3 16:02:29 2022] GET /identity/v3/role_assignments?group.id=ad54dac2940a42e194222806ee8b60e9&scope.project.id=23d03cc631264a3faa207c1b3bded8e3&role.id=a8e4174e3572497f9284c709a119dcff => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:02:29.977036 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7456044c-c4c9-40e5-a676-772c6ee8c835 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:29.978542 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7456044c-c4c9-40e5-a676-772c6ee8c835 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:30.018998 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:30.089508 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 221/452] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:02:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 16:02:30.095113 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f860dd3-0662-4c3c-bc6f-e52410279e01 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:30.174472 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:30.197986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f860dd3-0662-4c3c-bc6f-e52410279e01 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:30.234127 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb2456-8673-41d2-899e-c9a4073a0bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:30.234999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb2456-8673-41d2-899e-c9a4073a0bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:30.235277 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb2456-8673-41d2-899e-c9a4073a0bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:30.235616 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb2456-8673-41d2-899e-c9a4073a0bc0 None admin] PATH_INFO: `/v3/projects/450b64d0f10f4cb8949a26d9eea4aef4` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:30.236792 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cb2456-8673-41d2-899e-c9a4073a0bc0 None admin] RBAC: Authorizing `identity:get_project(project_id=450b64d0f10f4cb8949a26d9eea4aef4)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:30.238236 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cb2456-8673-41d2-899e-c9a4073a0bc0 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:30.240488 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 232/453] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:02:30 2022] GET /identity/v3/projects/450b64d0f10f4cb8949a26d9eea4aef4 => generated 344 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:30.246057 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7456044c-c4c9-40e5-a676-772c6ee8c835 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:30.347080 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7456044c-c4c9-40e5-a676-772c6ee8c835 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:30.381415 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52febcd-feab-4d98-9c7c-081f810d0f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:30.382272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52febcd-feab-4d98-9c7c-081f810d0f89 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:30.382555 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52febcd-feab-4d98-9c7c-081f810d0f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:30.382829 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52febcd-feab-4d98-9c7c-081f810d0f89 None admin] PATH_INFO: `/v3/groups/06ee27f162d145e79244190e976f5e0d` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:30.384020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52febcd-feab-4d98-9c7c-081f810d0f89 None admin] RBAC: Authorizing `identity:get_group(group_id=06ee27f162d145e79244190e976f5e0d)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:30.385465 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52febcd-feab-4d98-9c7c-081f810d0f89 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:30.387086 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 222/454] 10.222.0.41 () {56 vars in 1180 bytes} [Fri Jun 3 16:02:30 2022] GET /identity/v3/groups/06ee27f162d145e79244190e976f5e0d => generated 229 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:30.392848 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb2456-8673-41d2-899e-c9a4073a0bc0 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:30.493664 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb2456-8673-41d2-899e-c9a4073a0bc0 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:30.525992 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a1825-bd2a-4a8c-9295-e452c56549dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:30.526630 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a1825-bd2a-4a8c-9295-e452c56549dd None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:30.526911 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a1825-bd2a-4a8c-9295-e452c56549dd None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:30.527181 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a1825-bd2a-4a8c-9295-e452c56549dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:30.528034 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a1825-bd2a-4a8c-9295-e452c56549dd None admin] RBAC: Adding query filter params (group.id=06ee27f162d145e79244190e976f5e0d, role.id=75936a608b8149099ff54e317f3a6740, scope.project.id=450b64d0f10f4cb8949a26d9eea4aef4) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:30.528353 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a1825-bd2a-4a8c-9295-e452c56549dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:30.530014 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a1825-bd2a-4a8c-9295-e452c56549dd None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:30.535619 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 233/455] 10.222.0.41 () {56 vars in 1376 bytes} [Fri Jun 3 16:02:30 2022] GET /identity/v3/role_assignments?group.id=06ee27f162d145e79244190e976f5e0d&scope.project.id=450b64d0f10f4cb8949a26d9eea4aef4&role.id=75936a608b8149099ff54e317f3a6740 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:33.421916 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff49401-99b4-4b12-a333-5679f4aeab19 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:33.422861 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff49401-99b4-4b12-a333-5679f4aeab19 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:33.423362 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff49401-99b4-4b12-a333-5679f4aeab19 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:33.425143 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 223/456] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:33.430746 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c851a88-ebae-431b-bfa1-1b6d90a4a44f None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:33.431619 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c851a88-ebae-431b-bfa1-1b6d90a4a44f None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:33.432093 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c851a88-ebae-431b-bfa1-1b6d90a4a44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:33.460273 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-2c851a88-ebae-431b-bfa1-1b6d90a4a44f None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:33.461805 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-2c851a88-ebae-431b-bfa1-1b6d90a4a44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:33.464022 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:33.561647 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 234/457] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 16:02:33.598977 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff49401-99b4-4b12-a333-5679f4aeab19 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:33.603673 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:33.655833 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cbcaf9-c11a-4dce-9ea2-7edc613ac825 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:33.656463 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cbcaf9-c11a-4dce-9ea2-7edc613ac825 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:33.656936 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cbcaf9-c11a-4dce-9ea2-7edc613ac825 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:33.658750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 235/458] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:33.662903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07761a18-925c-4ebc-9db2-a1ca40ba53e5 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:33.663476 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07761a18-925c-4ebc-9db2-a1ca40ba53e5 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:33.663974 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07761a18-925c-4ebc-9db2-a1ca40ba53e5 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:33.665630 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 236/459] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:33.666843 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca6ed5-4b48-4e07-8aed-6cd9dd779d7a None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:33.667355 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca6ed5-4b48-4e07-8aed-6cd9dd779d7a None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:33.667825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca6ed5-4b48-4e07-8aed-6cd9dd779d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:33.690249 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-70ca6ed5-4b48-4e07-8aed-6cd9dd779d7a None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:33.691840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-70ca6ed5-4b48-4e07-8aed-6cd9dd779d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:33.722122 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff49401-99b4-4b12-a333-5679f4aeab19 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:33.760014 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1246992-32c8-4ffd-9f21-ea700b3cfca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:33.760931 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 237/460] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:33.761059 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1246992-32c8-4ffd-9f21-ea700b3cfca4 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:33.767313 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1246992-32c8-4ffd-9f21-ea700b3cfca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:33.767313 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1246992-32c8-4ffd-9f21-ea700b3cfca4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:33.768750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802a82c-4e8a-4533-8081-83a344bec30e None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:33.769190 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802a82c-4e8a-4533-8081-83a344bec30e None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:33.769664 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802a82c-4e8a-4533-8081-83a344bec30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:33.772371 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1246992-32c8-4ffd-9f21-ea700b3cfca4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:33.773915 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1246992-32c8-4ffd-9f21-ea700b3cfca4 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:33.794576 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-d1246992-32c8-4ffd-9f21-ea700b3cfca4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:33.794576 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:33.794576 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:33.794576 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:33.794576 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:33.794576 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:33.794576 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:33.794576 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:33.794576 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:33.794576 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:33.794576 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:33.794576 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:33.794576 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:33.794576 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:33.794576 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:33.795670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:33.795670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:33.795670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:33.795670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:33.795670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:33.795670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:33.795670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:33.795670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:33.795670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:33.795670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:33.795670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:33.795670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:33.795670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:33.795670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:33.795670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:33.795670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:33.795670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:33.796559 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:33.796559 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:33.796559 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:33.796559 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:33.798336 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 224/461] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:33.804331 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-e802a82c-4e8a-4533-8081-83a344bec30e None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:33.804637 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1246992-32c8-4ffd-9f21-ea700b3cfca4 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:33.805783 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-e802a82c-4e8a-4533-8081-83a344bec30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:33.871220 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:33.879460 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:33.887586 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 238/462] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:33.888409 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e802a82c-4e8a-4533-8081-83a344bec30e None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:33.927828 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1246992-32c8-4ffd-9f21-ea700b3cfca4 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:33.966387 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0af89f-0ec6-42b8-942f-42a51f224d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:33.967465 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0af89f-0ec6-42b8-942f-42a51f224d25 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:33.967820 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0af89f-0ec6-42b8-942f-42a51f224d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:33.968224 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0af89f-0ec6-42b8-942f-42a51f224d25 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:33.969060 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0af89f-0ec6-42b8-942f-42a51f224d25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:33.969363 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0af89f-0ec6-42b8-942f-42a51f224d25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:33.970751 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0af89f-0ec6-42b8-942f-42a51f224d25 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:33.985648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 225/463] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:33.992222 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3b454-62f4-4b99-bcec-39b65c6c7da6 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:33.992628 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3b454-62f4-4b99-bcec-39b65c6c7da6 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:33.993005 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3b454-62f4-4b99-bcec-39b65c6c7da6 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:33.998903 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 226/464] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:33.998903 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda3b454-62f4-4b99-bcec-39b65c6c7da6 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:34.009729 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e802a82c-4e8a-4533-8081-83a344bec30e None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:34.046386 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f893f4-e972-4226-8f40-eccd4d89f8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:34.047380 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f893f4-e972-4226-8f40-eccd4d89f8c4 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:34.047672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f893f4-e972-4226-8f40-eccd4d89f8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:34.048250 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f893f4-e972-4226-8f40-eccd4d89f8c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:34.060027 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f893f4-e972-4226-8f40-eccd4d89f8c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:34.061565 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f893f4-e972-4226-8f40-eccd4d89f8c4 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:34.081438 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-29f893f4-e972-4226-8f40-eccd4d89f8c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:34.081438 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:34.081438 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:34.081438 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:34.081438 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:34.081438 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:34.081438 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:34.081438 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:34.081438 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:34.081438 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:34.081438 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:34.081438 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:34.081438 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:34.081438 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:34.081438 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:34.082369 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:34.082369 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:34.082369 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:34.082369 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:34.082369 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:34.082369 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:34.082369 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:34.082369 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:34.082369 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:34.082369 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:34.082369 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:34.082369 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:34.082369 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:34.082369 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:34.082369 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:34.082369 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:34.082369 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:34.083221 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:34.083221 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:34.083221 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:34.083221 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:34.083221 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 239/465] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:34.083934 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f893f4-e972-4226-8f40-eccd4d89f8c4 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:34.106160 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda3b454-62f4-4b99-bcec-39b65c6c7da6 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:34.122061 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:34.142536 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:34.144536 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764cfc4d-d83b-4bc0-853d-7f3d51df86de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:34.145330 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764cfc4d-d83b-4bc0-853d-7f3d51df86de None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:34.145614 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764cfc4d-d83b-4bc0-853d-7f3d51df86de None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:34.145886 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764cfc4d-d83b-4bc0-853d-7f3d51df86de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:34.156581 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764cfc4d-d83b-4bc0-853d-7f3d51df86de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:34.158037 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764cfc4d-d83b-4bc0-853d-7f3d51df86de None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:34.174388 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-764cfc4d-d83b-4bc0-853d-7f3d51df86de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:34.174388 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:34.174388 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:34.174388 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:34.174388 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:34.174388 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:34.174388 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:34.174388 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:34.174388 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:34.174388 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:34.174388 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:34.174388 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:34.174388 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:34.174388 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:34.174388 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:34.175363 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:34.175363 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:34.175363 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:34.175363 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:34.175363 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:34.175363 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:34.175363 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:34.175363 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:34.175363 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:34.175363 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:34.175363 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:34.175363 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:34.175363 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:34.175363 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:34.175363 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:34.175363 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:34.175363 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:34.176727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:34.176727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:34.176727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:34.176727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:34.176727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 227/466] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:34.177378 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f5691-66aa-486e-b721-16fc5aa8928d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:34.177535 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f5691-66aa-486e-b721-16fc5aa8928d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:34.177798 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f5691-66aa-486e-b721-16fc5aa8928d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:34.196101 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-f22f5691-66aa-486e-b721-16fc5aa8928d None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:34.197273 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f22f5691-66aa-486e-b721-16fc5aa8928d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:34.200323 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f893f4-e972-4226-8f40-eccd4d89f8c4 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:34.241133 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e9b2d-6ebc-4f24-a684-43c5bb6176ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:34.241929 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e9b2d-6ebc-4f24-a684-43c5bb6176ea None admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:34.242217 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e9b2d-6ebc-4f24-a684-43c5bb6176ea None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:34.242497 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e9b2d-6ebc-4f24-a684-43c5bb6176ea None admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:34.243209 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87e9b2d-6ebc-4f24-a684-43c5bb6176ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:34.244794 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87e9b2d-6ebc-4f24-a684-43c5bb6176ea None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:34.279331 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 240/467] 10.222.0.41 () {60 vars in 1132 bytes} [Fri Jun 3 16:02:34 2022] POST /identity/v3/users => generated 246 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:02:34.279331 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e9b2d-6ebc-4f24-a684-43c5bb6176ea None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:34.282392 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 228/468] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:34.283393 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f5691-66aa-486e-b721-16fc5aa8928d None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:34.421025 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e9b2d-6ebc-4f24-a684-43c5bb6176ea None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:34.429694 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:34.441904 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:34.450776 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f5691-66aa-486e-b721-16fc5aa8928d None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:34.460836 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc577a-71bd-41b1-82dd-012f7176e3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:34.461583 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc577a-71bd-41b1-82dd-012f7176e3ac None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:34.461865 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc577a-71bd-41b1-82dd-012f7176e3ac None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:34.462189 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc577a-71bd-41b1-82dd-012f7176e3ac None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:34.463038 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fc577a-71bd-41b1-82dd-012f7176e3ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:34.463368 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fc577a-71bd-41b1-82dd-012f7176e3ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:34.464899 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fc577a-71bd-41b1-82dd-012f7176e3ac None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:34.483385 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 241/469] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:34.483385 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc577a-71bd-41b1-82dd-012f7176e3ac None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:34.496894 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db72b53-e193-497a-8179-92020e814426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:34.497554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db72b53-e193-497a-8179-92020e814426 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:34.497836 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db72b53-e193-497a-8179-92020e814426 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:34.498110 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db72b53-e193-497a-8179-92020e814426 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:34.499611 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db72b53-e193-497a-8179-92020e814426 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:34.499611 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db72b53-e193-497a-8179-92020e814426 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:34.500907 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db72b53-e193-497a-8179-92020e814426 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:34.512808 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 229/470] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:34.514074 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03440574-de04-4bf4-a357-715eea1d8ebb None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:34.514365 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03440574-de04-4bf4-a357-715eea1d8ebb None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:34.514640 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03440574-de04-4bf4-a357-715eea1d8ebb None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:34.515887 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 230/471] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:34.516622 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03440574-de04-4bf4-a357-715eea1d8ebb None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:34.593190 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc577a-71bd-41b1-82dd-012f7176e3ac None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:34.629637 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03440574-de04-4bf4-a357-715eea1d8ebb None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:34.636424 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b6c1c-6256-4e17-a3a8-b4156e20a02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:34.637231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b6c1c-6256-4e17-a3a8-b4156e20a02a None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:34.637510 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b6c1c-6256-4e17-a3a8-b4156e20a02a None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:34.637788 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b6c1c-6256-4e17-a3a8-b4156e20a02a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:34.648354 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b6c1c-6256-4e17-a3a8-b4156e20a02a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:34.649793 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b6c1c-6256-4e17-a3a8-b4156e20a02a None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:34.664699 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c885adc-d71d-4624-8a9a-6527eab8fb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:34.665355 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c885adc-d71d-4624-8a9a-6527eab8fb00 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:34.665636 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c885adc-d71d-4624-8a9a-6527eab8fb00 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:34.665911 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c885adc-d71d-4624-8a9a-6527eab8fb00 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:34.666608 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c885adc-d71d-4624-8a9a-6527eab8fb00 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:34.668063 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c885adc-d71d-4624-8a9a-6527eab8fb00 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:34.668336 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-c66b6c1c-6256-4e17-a3a8-b4156e20a02a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:34.668336 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:34.668336 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:34.668336 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:34.668336 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:34.668336 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:34.668336 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:34.668336 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:34.668336 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:34.668336 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:34.668336 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:34.668336 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:34.668336 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:34.668336 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:34.668336 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:34.669106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:34.669106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:34.669106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:34.669106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:34.669106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:34.669106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:34.669106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:34.669106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:34.669106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:34.669106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:34.669106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:34.669106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:34.669106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:34.669106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:34.669106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:34.669106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:34.669106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:34.669964 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:34.669964 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:34.669964 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:34.669964 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:34.669964 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 242/472] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:34.670703 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b6c1c-6256-4e17-a3a8-b4156e20a02a None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:34.690926 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:34.703538 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 231/473] 10.222.0.41 () {60 vars in 1132 bytes} [Fri Jun 3 16:02:34 2022] POST /identity/v3/users => generated 248 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:02:34.703538 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9fd1a-621b-4853-953f-21d384555c77 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:34.703538 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9fd1a-621b-4853-953f-21d384555c77 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:34.703538 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9fd1a-621b-4853-953f-21d384555c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:34.713778 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-97d9fd1a-621b-4853-953f-21d384555c77 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:34.714983 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-97d9fd1a-621b-4853-953f-21d384555c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:34.724001 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:34.801457 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 232/474] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:34.802514 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d9fd1a-621b-4853-953f-21d384555c77 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:34.831915 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b6c1c-6256-4e17-a3a8-b4156e20a02a None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:34.880172 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3239722-3a93-415b-a1d5-917282168c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:34.880862 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3239722-3a93-415b-a1d5-917282168c7e None admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:34.881148 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3239722-3a93-415b-a1d5-917282168c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:34.881426 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3239722-3a93-415b-a1d5-917282168c7e None admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:34.882160 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3239722-3a93-415b-a1d5-917282168c7e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:34.883720 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3239722-3a93-415b-a1d5-917282168c7e None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:34.909620 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 243/475] 10.222.0.41 () {60 vars in 1132 bytes} [Fri Jun 3 16:02:34 2022] POST /identity/v3/users => generated 248 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:02:34.911373 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3239722-3a93-415b-a1d5-917282168c7e None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:34.924785 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d9fd1a-621b-4853-953f-21d384555c77 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:34.965298 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d109c94c-0543-4324-8114-99871f736474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:34.965894 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109c94c-0543-4324-8114-99871f736474 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:34.966178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109c94c-0543-4324-8114-99871f736474 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:34.966458 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109c94c-0543-4324-8114-99871f736474 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:34.967203 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d109c94c-0543-4324-8114-99871f736474 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:34.968167 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d109c94c-0543-4324-8114-99871f736474 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:34.969617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d109c94c-0543-4324-8114-99871f736474 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:34.982691 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 233/476] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:34.989326 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d109c94c-0543-4324-8114-99871f736474 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:34.994624 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:35.001288 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:35.031159 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3239722-3a93-415b-a1d5-917282168c7e None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:35.071275 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5fba93-e239-4e78-bfbf-a03ea3096426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:35.071990 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5fba93-e239-4e78-bfbf-a03ea3096426 None admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:35.072278 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5fba93-e239-4e78-bfbf-a03ea3096426 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:35.072555 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5fba93-e239-4e78-bfbf-a03ea3096426 None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:35.073369 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5fba93-e239-4e78-bfbf-a03ea3096426 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:35.074759 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5fba93-e239-4e78-bfbf-a03ea3096426 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:35.082323 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.sql.core [None req-2b5fba93-e239-4e78-bfbf-a03ea3096426 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 03 16:02:35.082323 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 03 16:02:35.082323 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [parameters: {'id': '5c04c39f7a2e44ac885d23079354adc8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 03 16:02:35.082323 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80286) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 03 16:02:35.105186 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-2b5fba93-e239-4e78-bfbf-a03ea3096426 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 03 16:02:35.105186 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:35.105186 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 03 16:02:35.105186 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 03 16:02:35.105186 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 03 16:02:35.105186 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 03 16:02:35.105186 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 03 16:02:35.105186 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application result = self._query(query) Jun 03 16:02:35.105186 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 03 16:02:35.105186 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application conn.query(q) Jun 03 16:02:35.105186 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 03 16:02:35.105186 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 03 16:02:35.105186 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 03 16:02:35.105186 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application result.read() Jun 03 16:02:35.105186 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 03 16:02:35.106140 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 03 16:02:35.106140 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 03 16:02:35.106140 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 03 16:02:35.106140 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 03 16:02:35.106140 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 03 16:02:35.106140 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 03 16:02:35.106140 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 03 16:02:35.106140 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 03 16:02:35.106140 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:35.106140 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 03 16:02:35.106140 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:35.106140 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:35.106140 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 03 16:02:35.106140 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 03 16:02:35.106140 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 03 16:02:35.106140 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return ref.to_dict() Jun 03 16:02:35.106140 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 03 16:02:35.106140 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application next(self.gen) Jun 03 16:02:35.107047 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 03 16:02:35.107047 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application yield resource Jun 03 16:02:35.107047 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 03 16:02:35.107047 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application next(self.gen) Jun 03 16:02:35.107047 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 03 16:02:35.107047 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self.session.rollback() Jun 03 16:02:35.107047 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 03 16:02:35.107047 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self.force_reraise() Jun 03 16:02:35.107047 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 03 16:02:35.107047 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise self.value Jun 03 16:02:35.107047 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 03 16:02:35.107047 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 03 16:02:35.107047 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 03 16:02:35.107047 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application session.commit() Jun 03 16:02:35.107047 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit Jun 03 16:02:35.107047 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 03 16:02:35.107047 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 03 16:02:35.108023 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self._prepare_impl() Jun 03 16:02:35.108023 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 03 16:02:35.108023 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self.session.flush() Jun 03 16:02:35.108023 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush Jun 03 16:02:35.108023 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self._flush(objects) Jun 03 16:02:35.108023 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush Jun 03 16:02:35.108023 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 03 16:02:35.108023 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 03 16:02:35.108023 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application compat.raise_( Jun 03 16:02:35.108023 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ Jun 03 16:02:35.108023 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception Jun 03 16:02:35.108023 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush Jun 03 16:02:35.108023 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application flush_context.execute() Jun 03 16:02:35.108023 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 03 16:02:35.108023 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rec.execute(self) Jun 03 16:02:35.108023 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 03 16:02:35.108023 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 03 16:02:35.108873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 03 16:02:35.108873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 03 16:02:35.108873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 03 16:02:35.108873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 03 16:02:35.108873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 03 16:02:35.108873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 03 16:02:35.108873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 03 16:02:35.108873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 03 16:02:35.108873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 03 16:02:35.108873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 03 16:02:35.108873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 03 16:02:35.108873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 03 16:02:35.108873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 03 16:02:35.108873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 03 16:02:35.108873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ Jun 03 16:02:35.108873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception Jun 03 16:02:35.109677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 03 16:02:35.109677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 03 16:02:35.109677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 03 16:02:35.109677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 03 16:02:35.109677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 03 16:02:35.109677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application result = self._query(query) Jun 03 16:02:35.109677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 03 16:02:35.109677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application conn.query(q) Jun 03 16:02:35.109677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 03 16:02:35.109677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 03 16:02:35.109677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 03 16:02:35.109677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application result.read() Jun 03 16:02:35.109677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 03 16:02:35.109677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 03 16:02:35.109677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 03 16:02:35.109677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 03 16:02:35.110441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 03 16:02:35.110441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 03 16:02:35.110441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 03 16:02:35.110441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 03 16:02:35.110441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 03 16:02:35.110441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 03 16:02:35.110441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [parameters: {'id': '5c04c39f7a2e44ac885d23079354adc8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 03 16:02:35.110441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 03 16:02:35.110441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:35.110441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 16:02:35.110441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:35.110441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:35.110441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:35.110441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:35.110441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:35.110441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:35.111211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:35.111211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:35.111211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:35.111211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:35.111211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:35.111211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:35.111211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 03 16:02:35.111211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 03 16:02:35.111211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:35.111211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:35.111211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 03 16:02:35.111211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 03 16:02:35.111211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 03 16:02:35.111211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 03 16:02:35.111211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 03 16:02:35.111211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:35.117500 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 244/477] 10.222.0.41 () {60 vars in 1131 bytes} [Fri Jun 3 16:02:34 2022] POST /identity/v3/roles => generated 144 bytes in 197 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:35.117644 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d109c94c-0543-4324-8114-99871f736474 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:35.117793 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5fba93-e239-4e78-bfbf-a03ea3096426 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:35.154281 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf33468-fe48-4053-bbc8-af9df35344e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:35.155061 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf33468-fe48-4053-bbc8-af9df35344e1 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:35.155542 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf33468-fe48-4053-bbc8-af9df35344e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:35.155886 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf33468-fe48-4053-bbc8-af9df35344e1 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:35.156640 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf33468-fe48-4053-bbc8-af9df35344e1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:35.158228 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf33468-fe48-4053-bbc8-af9df35344e1 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:35.183419 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 234/478] 10.222.0.41 () {60 vars in 1132 bytes} [Fri Jun 3 16:02:34 2022] POST /identity/v3/users => generated 249 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:02:35.244835 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5fba93-e239-4e78-bfbf-a03ea3096426 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:35.283164 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b511ef7-8bdf-41aa-8522-9f7706c6e5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:35.284024 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b511ef7-8bdf-41aa-8522-9f7706c6e5cc None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:35.284310 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b511ef7-8bdf-41aa-8522-9f7706c6e5cc None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:35.284620 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b511ef7-8bdf-41aa-8522-9f7706c6e5cc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:35.293349 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:35.299367 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b511ef7-8bdf-41aa-8522-9f7706c6e5cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:35.312030 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-7b511ef7-8bdf-41aa-8522-9f7706c6e5cc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 16:02:35.312030 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:35.312030 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:35.312030 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:35.312030 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:35.312030 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:35.312030 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:35.312030 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:35.312030 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:35.312030 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:35.312030 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:35.312030 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:35.312030 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:35.312030 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:35.312030 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:35.312030 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:35.314584 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:35.314584 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:35.314584 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:35.314584 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:35.314584 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:35.314584 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:35.314584 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:35.314584 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:35.314584 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:35.314584 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:35.314584 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:35.314584 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:35.314584 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:35.314584 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:35.314584 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:35.314584 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:35.317259 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:35.317259 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:35.317259 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:35.317259 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:35.317259 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:35.317259 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:35.317259 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:35.317259 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:35.317259 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:35.317259 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:35.317259 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 16:02:35.317259 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:35.317259 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 245/479] 10.222.0.41 () {56 vars in 1114 bytes} [Fri Jun 3 16:02:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:35.322983 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf33468-fe48-4053-bbc8-af9df35344e1 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:35.393870 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:35.443379 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf33468-fe48-4053-bbc8-af9df35344e1 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:35.480723 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c5da4f-b61e-4747-9f45-2daf443c0bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:35.481394 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5da4f-b61e-4747-9f45-2daf443c0bbd None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:35.481678 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5da4f-b61e-4747-9f45-2daf443c0bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:35.481951 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5da4f-b61e-4747-9f45-2daf443c0bbd None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:35.482703 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c5da4f-b61e-4747-9f45-2daf443c0bbd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:35.483015 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c5da4f-b61e-4747-9f45-2daf443c0bbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:35.484447 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c5da4f-b61e-4747-9f45-2daf443c0bbd None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:35.494389 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 235/480] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:02:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:36.817915 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a860343c-af3c-46b8-8d8b-afe6c13e1d13 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:36.818623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a860343c-af3c-46b8-8d8b-afe6c13e1d13 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:36.818623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a860343c-af3c-46b8-8d8b-afe6c13e1d13 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:36.820104 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 246/481] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:36.825477 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e7571-9de2-45bc-afe8-f3c7fb3bc484 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:36.825944 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e7571-9de2-45bc-afe8-f3c7fb3bc484 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:36.826092 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e7571-9de2-45bc-afe8-f3c7fb3bc484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:36.852271 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-697e7571-9de2-45bc-afe8-f3c7fb3bc484 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:36.853477 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-697e7571-9de2-45bc-afe8-f3c7fb3bc484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:36.945548 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 236/482] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 16:02:36.990105 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a860343c-af3c-46b8-8d8b-afe6c13e1d13 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:37.250874 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a860343c-af3c-46b8-8d8b-afe6c13e1d13 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:37.298781 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd4636-f650-4c94-b58d-42d53dca4713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:37.299615 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd4636-f650-4c94-b58d-42d53dca4713 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:37.299901 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd4636-f650-4c94-b58d-42d53dca4713 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:37.300178 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd4636-f650-4c94-b58d-42d53dca4713 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:37.321367 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadd4636-f650-4c94-b58d-42d53dca4713 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:37.329162 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-eadd4636-f650-4c94-b58d-42d53dca4713 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:02:37.329162 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:37.329162 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:37.329162 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:37.329162 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:37.329162 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:37.329162 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:37.329162 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:37.329162 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:37.329162 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:37.329162 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:37.329162 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:37.329162 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:37.329162 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:37.329162 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:37.329162 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:37.331141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:37.331141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:37.331141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:37.331141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:37.331141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:37.331141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:37.331141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:37.331141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:37.331141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:37.331141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:37.331141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:37.331141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:37.331141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:37.331141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:37.331141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:37.331141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:37.333669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:37.333669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:37.333669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:37.333669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:37.333669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:37.333669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:37.333669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:37.333669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:37.333669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:37.333669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:37.333669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:02:37.333669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:37.339330 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 247/483] 10.222.0.41 () {56 vars in 1102 bytes} [Fri Jun 3 16:02:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:37.339613 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e7571-9de2-45bc-afe8-f3c7fb3bc484 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:37.431684 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:37.501120 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e7571-9de2-45bc-afe8-f3c7fb3bc484 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:37.551364 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b51033-7aed-46aa-8852-b195266ff067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:37.552109 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b51033-7aed-46aa-8852-b195266ff067 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:37.552400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b51033-7aed-46aa-8852-b195266ff067 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:37.552677 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b51033-7aed-46aa-8852-b195266ff067 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:37.553445 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b51033-7aed-46aa-8852-b195266ff067 None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:37.553756 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b51033-7aed-46aa-8852-b195266ff067 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:37.555146 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b51033-7aed-46aa-8852-b195266ff067 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:37.565057 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 237/484] 10.222.0.41 () {56 vars in 1103 bytes} [Fri Jun 3 16:02:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:37.570416 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd4636-f650-4c94-b58d-42d53dca4713 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:37.651482 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:37.695097 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd4636-f650-4c94-b58d-42d53dca4713 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:37.721328 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc138fae-d405-4462-b7b7-e8f4b12d731e None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:37.721628 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc138fae-d405-4462-b7b7-e8f4b12d731e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:37.721903 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc138fae-d405-4462-b7b7-e8f4b12d731e None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:37.723867 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 238/485] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:37.729340 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c22c14-885d-423a-be9d-42ad512bc518 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:37.729906 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c22c14-885d-423a-be9d-42ad512bc518 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:37.730282 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c22c14-885d-423a-be9d-42ad512bc518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:37.731746 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e91e7a9-596b-4159-89d0-44f90c00c016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:37.732531 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91e7a9-596b-4159-89d0-44f90c00c016 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:37.732818 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91e7a9-596b-4159-89d0-44f90c00c016 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:37.733097 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91e7a9-596b-4159-89d0-44f90c00c016 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:37.747408 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e91e7a9-596b-4159-89d0-44f90c00c016 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:37.747408 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e91e7a9-596b-4159-89d0-44f90c00c016 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:37.754580 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-92c22c14-885d-423a-be9d-42ad512bc518 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:37.755737 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-92c22c14-885d-423a-be9d-42ad512bc518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:37.780266 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-9e91e7a9-596b-4159-89d0-44f90c00c016 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:37.780266 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:37.780266 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:37.780266 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:37.780266 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:37.780266 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:37.780266 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:37.780266 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:37.780266 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:37.780266 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:37.780266 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:37.780266 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:37.780266 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:37.780266 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:37.780266 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:37.781168 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:37.781168 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:37.781168 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:37.781168 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:37.781168 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:37.781168 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:37.781168 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:37.781168 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:37.781168 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:37.781168 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:37.781168 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:37.781168 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:37.781168 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:37.781168 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:37.781168 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:37.781168 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:37.781168 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:37.782280 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:37.782280 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:37.782280 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:37.782280 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:37.782280 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 248/486] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:37.789288 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e91e7a9-596b-4159-89d0-44f90c00c016 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:37.891167 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 239/487] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 16:02:37.926245 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e91e7a9-596b-4159-89d0-44f90c00c016 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:37.935547 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97a6df-0bb3-4665-a1ee-421c217d7112 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:37.935838 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97a6df-0bb3-4665-a1ee-421c217d7112 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:37.936119 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97a6df-0bb3-4665-a1ee-421c217d7112 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:37.937739 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 240/488] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:37.938467 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb97a6df-0bb3-4665-a1ee-421c217d7112 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:37.939592 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:37.972078 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c41ee1-be5b-4b10-af54-9f0ad873bd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:37.972813 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c41ee1-be5b-4b10-af54-9f0ad873bd2c None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:37.973098 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c41ee1-be5b-4b10-af54-9f0ad873bd2c None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:37.973377 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c41ee1-be5b-4b10-af54-9f0ad873bd2c None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:37.974259 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c41ee1-be5b-4b10-af54-9f0ad873bd2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:37.974679 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c41ee1-be5b-4b10-af54-9f0ad873bd2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:37.976131 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c41ee1-be5b-4b10-af54-9f0ad873bd2c None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:37.992176 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 249/489] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:37.993720 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387497f4-c070-4a10-ac15-db1c1f19939c None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:37.994368 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387497f4-c070-4a10-ac15-db1c1f19939c None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:37.994654 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387497f4-c070-4a10-ac15-db1c1f19939c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:37.998769 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:38.014299 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-387497f4-c070-4a10-ac15-db1c1f19939c None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:38.015540 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-387497f4-c070-4a10-ac15-db1c1f19939c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:38.073758 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb97a6df-0bb3-4665-a1ee-421c217d7112 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:38.091603 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 250/490] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:38.092480 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387497f4-c070-4a10-ac15-db1c1f19939c None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:38.137028 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c577a-20ae-43fd-a7b1-d2cd42f91466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:38.137942 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c577a-20ae-43fd-a7b1-d2cd42f91466 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:38.138293 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c577a-20ae-43fd-a7b1-d2cd42f91466 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:38.138639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c577a-20ae-43fd-a7b1-d2cd42f91466 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:38.149678 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8c577a-20ae-43fd-a7b1-d2cd42f91466 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:38.157826 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-6b8c577a-20ae-43fd-a7b1-d2cd42f91466 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:02:38.157826 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:38.157826 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:38.157826 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:38.157826 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:38.157826 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:38.157826 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:38.157826 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:38.157826 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:38.157826 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:38.157826 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:38.157826 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:38.157826 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:38.157826 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:38.157826 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:38.157826 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:38.158681 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:38.158681 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:38.158681 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:38.158681 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:38.158681 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:38.158681 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:38.158681 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:38.158681 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:38.158681 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:38.158681 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:38.158681 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:38.158681 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:38.158681 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:38.158681 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:38.158681 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:38.158681 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:38.159703 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:38.159703 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:38.159703 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:38.159703 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:38.159703 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:38.159703 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:38.159703 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:38.159703 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:38.159703 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:38.159703 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:38.159703 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:02:38.159703 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:38.159703 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 241/491] 10.222.0.41 () {56 vars in 1102 bytes} [Fri Jun 3 16:02:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:38.160798 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c577a-20ae-43fd-a7b1-d2cd42f91466 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:38.199971 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:38.203877 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387497f4-c070-4a10-ac15-db1c1f19939c None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:38.248502 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37807d7-be24-44c2-8957-1b09d4caccd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:38.249648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37807d7-be24-44c2-8957-1b09d4caccd5 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:38.250147 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37807d7-be24-44c2-8957-1b09d4caccd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:38.250633 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37807d7-be24-44c2-8957-1b09d4caccd5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:38.259137 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37807d7-be24-44c2-8957-1b09d4caccd5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:38.260944 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37807d7-be24-44c2-8957-1b09d4caccd5 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:38.275840 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c577a-20ae-43fd-a7b1-d2cd42f91466 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:38.277474 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:38.279211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-a37807d7-be24-44c2-8957-1b09d4caccd5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 03 16:02:38.279211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:38.279211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:38.279211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:38.279211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:38.279211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:38.279211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:38.279211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:38.279211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:38.279211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:38.279211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:38.279211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:38.279211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:02:38.279211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:02:38.279211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:02:38.279211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:02:38.280099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:38.280099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:38.280099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:02:38.280099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:02:38.280099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:02:38.280099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:02:38.280099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:38.280099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:38.280099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:38.280099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:38.280099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:38.280099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:38.280099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:38.280099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:38.280099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:38.280099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:38.280099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:38.280099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:38.280977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:38.280977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:38.280977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:02:38.280977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:02:38.280977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:02:38.280977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:02:38.280977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:02:38.280977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:02:38.280977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 03 16:02:38.280977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:38.283605 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 251/492] 10.222.0.41 () {56 vars in 1093 bytes} [Fri Jun 3 16:02:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:38.284599 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37807d7-be24-44c2-8957-1b09d4caccd5 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:38.325575 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630012e6-a9db-424c-93be-08b61b79c71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:38.326378 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630012e6-a9db-424c-93be-08b61b79c71d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:38.326663 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630012e6-a9db-424c-93be-08b61b79c71d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:38.326947 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630012e6-a9db-424c-93be-08b61b79c71d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:38.338192 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630012e6-a9db-424c-93be-08b61b79c71d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:38.345834 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-630012e6-a9db-424c-93be-08b61b79c71d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:02:38.345834 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:38.345834 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:38.345834 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:38.345834 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:38.345834 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:38.345834 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:38.345834 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:38.345834 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:38.345834 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:38.345834 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:38.345834 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:38.345834 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:38.345834 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:38.345834 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:38.345834 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:38.346787 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:38.346787 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:38.346787 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:38.346787 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:38.346787 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:38.346787 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:38.346787 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:38.346787 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:38.346787 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:38.346787 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:38.346787 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:38.346787 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:38.346787 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:38.346787 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:38.346787 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:38.346787 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:38.347997 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:38.347997 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:38.347997 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:38.347997 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:38.347997 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:38.347997 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:38.347997 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:38.347997 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:38.347997 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:38.347997 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:38.347997 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:02:38.347997 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:38.347997 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 242/493] 10.222.0.41 () {56 vars in 1102 bytes} [Fri Jun 3 16:02:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:38.347997 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630012e6-a9db-424c-93be-08b61b79c71d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:38.399031 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37807d7-be24-44c2-8957-1b09d4caccd5 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:38.432971 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4cca19-96ba-40e1-af02-3cf6f55b73fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:38.433638 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4cca19-96ba-40e1-af02-3cf6f55b73fc None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:38.433932 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4cca19-96ba-40e1-af02-3cf6f55b73fc None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:38.434214 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4cca19-96ba-40e1-af02-3cf6f55b73fc None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:38.434962 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4cca19-96ba-40e1-af02-3cf6f55b73fc None admin] RBAC: Adding query filter params (name=service) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:38.435273 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4cca19-96ba-40e1-af02-3cf6f55b73fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:38.436653 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4cca19-96ba-40e1-af02-3cf6f55b73fc None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:38.449450 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 252/494] 10.222.0.41 () {56 vars in 1103 bytes} [Fri Jun 3 16:02:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:38.449450 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802df5a-355a-46f0-8d99-b1b12dd8e105 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:38.449450 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802df5a-355a-46f0-8d99-b1b12dd8e105 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:38.449450 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802df5a-355a-46f0-8d99-b1b12dd8e105 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:38.449450 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 253/495] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:38.449838 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6802df5a-355a-46f0-8d99-b1b12dd8e105 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:38.458627 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630012e6-a9db-424c-93be-08b61b79c71d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:38.474979 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:38.492612 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b56d3a-bced-4204-99f8-4ae044adda28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:38.493270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b56d3a-bced-4204-99f8-4ae044adda28 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:38.493555 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b56d3a-bced-4204-99f8-4ae044adda28 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:38.493837 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b56d3a-bced-4204-99f8-4ae044adda28 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:38.494676 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b56d3a-bced-4204-99f8-4ae044adda28 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:38.494988 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b56d3a-bced-4204-99f8-4ae044adda28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:38.496479 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b56d3a-bced-4204-99f8-4ae044adda28 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:38.533107 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 243/496] 10.222.0.41 () {56 vars in 1133 bytes} [Fri Jun 3 16:02:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:38.534214 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b56d3a-bced-4204-99f8-4ae044adda28 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:38.555172 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:38.555374 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6802df5a-355a-46f0-8d99-b1b12dd8e105 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:38.588830 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b24a3-0dd2-4f90-9fea-8747814b1fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:38.589487 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b24a3-0dd2-4f90-9fea-8747814b1fc3 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:38.589771 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b24a3-0dd2-4f90-9fea-8747814b1fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:38.590042 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b24a3-0dd2-4f90-9fea-8747814b1fc3 None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:38.590774 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97b24a3-0dd2-4f90-9fea-8747814b1fc3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:38.591080 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97b24a3-0dd2-4f90-9fea-8747814b1fc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:38.592474 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97b24a3-0dd2-4f90-9fea-8747814b1fc3 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:38.600811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 254/497] 10.222.0.41 () {56 vars in 1103 bytes} [Fri Jun 3 16:02:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:38.601989 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b283581-05fa-4b5a-a500-abbe113e504b None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:38.602278 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b283581-05fa-4b5a-a500-abbe113e504b None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:38.602554 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b283581-05fa-4b5a-a500-abbe113e504b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:38.617974 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-5b283581-05fa-4b5a-a500-abbe113e504b None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:38.619150 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-5b283581-05fa-4b5a-a500-abbe113e504b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:38.634152 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b56d3a-bced-4204-99f8-4ae044adda28 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:38.667801 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a59b93e-0747-4d14-b884-f221d78b3625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:38.668611 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59b93e-0747-4d14-b884-f221d78b3625 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:38.668891 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59b93e-0747-4d14-b884-f221d78b3625 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:38.669163 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59b93e-0747-4d14-b884-f221d78b3625 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:38.678722 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a59b93e-0747-4d14-b884-f221d78b3625 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:38.680191 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a59b93e-0747-4d14-b884-f221d78b3625 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:38.683542 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 255/498] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:38.684808 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c3c99e-1221-4cb0-95bc-84e5c989109e None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:38.685097 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c3c99e-1221-4cb0-95bc-84e5c989109e None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:38.685373 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c3c99e-1221-4cb0-95bc-84e5c989109e None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:38.686610 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 256/499] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:38.687316 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c3c99e-1221-4cb0-95bc-84e5c989109e None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:38.697577 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-7a59b93e-0747-4d14-b884-f221d78b3625 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:38.697577 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:38.697577 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:38.697577 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:38.697577 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:38.697577 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:38.697577 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:38.697577 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:38.697577 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:38.697577 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:38.697577 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:38.697577 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:38.697577 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:38.697577 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:38.697577 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:38.698431 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:38.698431 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:38.698431 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:38.698431 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:38.698431 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:38.698431 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:38.698431 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:38.698431 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:38.698431 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:38.698431 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:38.698431 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:38.698431 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:38.698431 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:38.698431 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:38.698431 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:38.698431 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:38.698431 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:38.699317 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:38.699317 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:38.699317 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:38.699317 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:38.703223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 244/500] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:38.704077 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a59b93e-0747-4d14-b884-f221d78b3625 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:38.705279 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:38.773406 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:38.790733 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c3c99e-1221-4cb0-95bc-84e5c989109e None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:38.804817 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a59b93e-0747-4d14-b884-f221d78b3625 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:38.832519 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b931d2-2c4e-4ef3-8186-71e3ffe90de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:38.833159 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b931d2-2c4e-4ef3-8186-71e3ffe90de5 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:38.833445 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b931d2-2c4e-4ef3-8186-71e3ffe90de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:38.833717 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b931d2-2c4e-4ef3-8186-71e3ffe90de5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:38.837336 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b7d884-8b2f-4aa6-bdec-16544d587175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:38.838107 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7d884-8b2f-4aa6-bdec-16544d587175 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:38.838391 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7d884-8b2f-4aa6-bdec-16544d587175 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:38.838662 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7d884-8b2f-4aa6-bdec-16544d587175 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:38.850406 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b7d884-8b2f-4aa6-bdec-16544d587175 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:38.851901 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b7d884-8b2f-4aa6-bdec-16544d587175 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:38.868819 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-c6b7d884-8b2f-4aa6-bdec-16544d587175 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:38.868819 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:38.868819 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:38.868819 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:38.868819 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:38.868819 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:38.868819 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:38.868819 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:38.868819 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:38.868819 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:38.868819 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:38.868819 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:38.868819 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:38.868819 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:38.868819 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:38.869640 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:38.869640 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:38.869640 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:38.869640 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:38.869640 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:38.869640 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:38.869640 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:38.869640 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:38.869640 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:38.869640 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:38.869640 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:38.869640 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:38.869640 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:38.869640 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:38.869640 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:38.869640 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:38.869640 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:38.870495 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:38.870495 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:38.870495 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:38.870495 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:38.870495 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 245/501] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:38.871510 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e65e16-b7e9-4ae7-aef2-b697d483954a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:38.871800 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e65e16-b7e9-4ae7-aef2-b697d483954a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:38.872076 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e65e16-b7e9-4ae7-aef2-b697d483954a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:38.905292 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b931d2-2c4e-4ef3-8186-71e3ffe90de5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:38.906761 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b931d2-2c4e-4ef3-8186-71e3ffe90de5 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:38.924120 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:39.011404 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-a7e65e16-b7e9-4ae7-aef2-b697d483954a None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:39.012628 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a7e65e16-b7e9-4ae7-aef2-b697d483954a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:39.017086 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 257/502] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:02:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 16:02:39.017918 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b931d2-2c4e-4ef3-8186-71e3ffe90de5 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:39.080229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 246/503] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:39.080993 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e65e16-b7e9-4ae7-aef2-b697d483954a None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:39.119765 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b931d2-2c4e-4ef3-8186-71e3ffe90de5 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:39.133162 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:39.134012 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:39.152542 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d4342-6b7c-4820-a9c7-7362ccbb90b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:39.153231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d4342-6b7c-4820-a9c7-7362ccbb90b8 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:39.153521 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d4342-6b7c-4820-a9c7-7362ccbb90b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:39.153796 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d4342-6b7c-4820-a9c7-7362ccbb90b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:39.154766 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d4342-6b7c-4820-a9c7-7362ccbb90b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:39.155078 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d4342-6b7c-4820-a9c7-7362ccbb90b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:39.156467 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d4342-6b7c-4820-a9c7-7362ccbb90b8 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:39.167844 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 258/504] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:39.168614 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d4342-6b7c-4820-a9c7-7362ccbb90b8 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:39.177643 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e65e16-b7e9-4ae7-aef2-b697d483954a None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:39.210637 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72cb8dc-f6fe-4177-a339-73554c8f58f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:39.211450 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cb8dc-f6fe-4177-a339-73554c8f58f8 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:39.211730 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cb8dc-f6fe-4177-a339-73554c8f58f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:39.212004 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cb8dc-f6fe-4177-a339-73554c8f58f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:39.221614 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72cb8dc-f6fe-4177-a339-73554c8f58f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:39.223635 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72cb8dc-f6fe-4177-a339-73554c8f58f8 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:39.241255 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-c72cb8dc-f6fe-4177-a339-73554c8f58f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:39.241255 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:39.241255 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:39.241255 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:39.241255 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:39.241255 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:39.241255 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:39.241255 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:39.241255 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:39.241255 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:39.241255 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:39.241255 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:39.241255 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:39.241255 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:39.241255 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:39.242175 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:39.242175 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:39.242175 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:39.242175 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:39.242175 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:39.242175 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:39.242175 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:39.242175 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:39.242175 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:39.242175 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:39.242175 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:39.242175 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:39.242175 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:39.242175 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:39.242175 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:39.242175 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:39.242175 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:39.243669 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:39.243669 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:39.243669 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:39.243669 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:39.243669 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 247/505] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:39.243669 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72cb8dc-f6fe-4177-a339-73554c8f58f8 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:39.270709 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d4342-6b7c-4820-a9c7-7362ccbb90b8 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:39.306048 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3396025-6a27-45fa-8ea7-76e25dd54e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:39.306714 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3396025-6a27-45fa-8ea7-76e25dd54e39 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:39.307137 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3396025-6a27-45fa-8ea7-76e25dd54e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:39.307588 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3396025-6a27-45fa-8ea7-76e25dd54e39 None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:39.308029 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3396025-6a27-45fa-8ea7-76e25dd54e39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:39.308338 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3396025-6a27-45fa-8ea7-76e25dd54e39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:39.309708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3396025-6a27-45fa-8ea7-76e25dd54e39 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:39.322200 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 259/506] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:39.323015 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3396025-6a27-45fa-8ea7-76e25dd54e39 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:39.349162 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72cb8dc-f6fe-4177-a339-73554c8f58f8 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:39.355960 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:39.358650 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:39.383112 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67087458-b2c7-4a84-8ada-62326057b62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:39.383928 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67087458-b2c7-4a84-8ada-62326057b62e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:39.384215 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67087458-b2c7-4a84-8ada-62326057b62e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:39.384495 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67087458-b2c7-4a84-8ada-62326057b62e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:39.394824 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67087458-b2c7-4a84-8ada-62326057b62e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:39.396302 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67087458-b2c7-4a84-8ada-62326057b62e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:39.414156 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-67087458-b2c7-4a84-8ada-62326057b62e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:39.414156 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:39.414156 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:39.414156 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:39.414156 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:39.414156 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:39.414156 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:39.414156 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:39.414156 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:39.414156 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:39.414156 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:39.414156 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:39.414156 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:39.414156 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:39.414156 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:39.414980 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:39.414980 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:39.414980 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:39.414980 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:39.414980 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:39.414980 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:39.414980 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:39.414980 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:39.414980 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:39.414980 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:39.414980 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:39.414980 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:39.414980 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:39.414980 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:39.414980 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:39.414980 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:39.414980 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:39.416244 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:39.416244 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:39.416244 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:39.416244 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:39.416244 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 248/507] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:39.416244 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67087458-b2c7-4a84-8ada-62326057b62e None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:39.427518 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3396025-6a27-45fa-8ea7-76e25dd54e39 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:39.461541 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed816e79-3071-4bd9-b570-3c1b6c8ee83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:39.462330 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed816e79-3071-4bd9-b570-3c1b6c8ee83b None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:39.462615 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed816e79-3071-4bd9-b570-3c1b6c8ee83b None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:39.462897 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed816e79-3071-4bd9-b570-3c1b6c8ee83b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:39.473130 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed816e79-3071-4bd9-b570-3c1b6c8ee83b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:39.481012 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-ed816e79-3071-4bd9-b570-3c1b6c8ee83b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:02:39.481012 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:39.481012 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:39.481012 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:39.481012 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:39.481012 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:39.481012 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:39.481012 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:39.481012 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:39.481012 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:39.481012 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:39.481012 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:39.481012 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:39.481012 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:39.481012 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:39.481012 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:39.481954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:39.481954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:39.481954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:39.481954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:39.481954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:39.481954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:39.481954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:39.481954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:39.481954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:39.481954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:39.481954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:39.481954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:39.481954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:39.481954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:39.481954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:39.481954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:39.482751 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:39.482751 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:39.482751 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:39.482751 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:39.482751 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:39.482751 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:39.482751 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:39.482751 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:39.482751 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:39.482751 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:39.482751 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:02:39.482751 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:39.482751 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 260/508] 10.222.0.41 () {56 vars in 1102 bytes} [Fri Jun 3 16:02:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:39.483424 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed816e79-3071-4bd9-b570-3c1b6c8ee83b None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:39.517047 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67087458-b2c7-4a84-8ada-62326057b62e None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:39.552553 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d241a-e9b8-490d-894a-79d21a3bb2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:39.553359 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d241a-e9b8-490d-894a-79d21a3bb2c1 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:39.553642 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d241a-e9b8-490d-894a-79d21a3bb2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:39.553943 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d241a-e9b8-490d-894a-79d21a3bb2c1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:39.561971 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958d241a-e9b8-490d-894a-79d21a3bb2c1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:39.563431 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958d241a-e9b8-490d-894a-79d21a3bb2c1 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:39.575006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:39.578990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-958d241a-e9b8-490d-894a-79d21a3bb2c1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 03 16:02:39.578990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:39.578990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:39.578990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:39.578990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:39.578990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:39.578990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:39.578990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:39.578990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:39.578990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:39.578990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:39.578990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:39.578990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:02:39.578990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:02:39.578990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:02:39.578990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:02:39.579898 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:39.579898 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:39.579898 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:02:39.579898 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:02:39.579898 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:02:39.579898 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:02:39.579898 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:39.579898 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:39.579898 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:39.579898 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:39.579898 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:39.579898 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:39.579898 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:39.579898 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:39.579898 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:39.579898 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:39.579898 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:39.579898 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:39.580812 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:39.580812 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:39.580812 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:02:39.580812 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:02:39.580812 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:02:39.580812 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:02:39.580812 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:02:39.580812 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:02:39.580812 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 03 16:02:39.580812 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:39.580812 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 249/509] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:02:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:39.582381 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d241a-e9b8-490d-894a-79d21a3bb2c1 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:39.587405 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed816e79-3071-4bd9-b570-3c1b6c8ee83b None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:39.601359 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:39.620846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0f538-99ce-4b4b-92e1-4318a2b419ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:39.621507 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0f538-99ce-4b4b-92e1-4318a2b419ee None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:39.621791 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0f538-99ce-4b4b-92e1-4318a2b419ee None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:39.622070 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0f538-99ce-4b4b-92e1-4318a2b419ee None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:39.622794 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de0f538-99ce-4b4b-92e1-4318a2b419ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:39.623106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de0f538-99ce-4b4b-92e1-4318a2b419ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:39.624503 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de0f538-99ce-4b4b-92e1-4318a2b419ee None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:39.636489 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 261/510] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:39.637269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0f538-99ce-4b4b-92e1-4318a2b419ee None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:39.683992 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d241a-e9b8-490d-894a-79d21a3bb2c1 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:39.716738 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e502ff-c36d-4ba0-9c4b-c2d566ab53e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:39.717520 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e502ff-c36d-4ba0-9c4b-c2d566ab53e4 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:39.717802 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e502ff-c36d-4ba0-9c4b-c2d566ab53e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:39.718078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e502ff-c36d-4ba0-9c4b-c2d566ab53e4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:39.725382 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e502ff-c36d-4ba0-9c4b-c2d566ab53e4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:39.726814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e502ff-c36d-4ba0-9c4b-c2d566ab53e4 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:39.733890 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0f538-99ce-4b4b-92e1-4318a2b419ee None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:39.741709 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-e1e502ff-c36d-4ba0-9c4b-c2d566ab53e4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 03 16:02:39.741709 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:39.741709 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:39.741709 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:39.741709 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:39.741709 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:39.741709 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:39.741709 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:39.741709 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:39.741709 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:39.741709 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:39.741709 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:39.741709 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:02:39.741709 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:02:39.741709 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:02:39.741709 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:02:39.742591 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:39.742591 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:39.742591 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:02:39.742591 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:02:39.742591 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:02:39.742591 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:02:39.742591 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:39.742591 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:39.742591 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:39.742591 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:39.742591 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:39.742591 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:39.742591 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:39.742591 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:39.742591 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:39.742591 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:39.742591 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:39.742591 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:39.743670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:39.743670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:39.743670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:02:39.743670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:02:39.743670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:02:39.743670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:02:39.743670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:02:39.743670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:02:39.743670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 03 16:02:39.743670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:39.743670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 250/511] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:02:39 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:39.743670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e502ff-c36d-4ba0-9c4b-c2d566ab53e4 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:39.769225 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870e1c0-45de-441a-8661-0122f8f80971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:39.770460 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870e1c0-45de-441a-8661-0122f8f80971 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:39.770809 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870e1c0-45de-441a-8661-0122f8f80971 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:39.771089 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870e1c0-45de-441a-8661-0122f8f80971 None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:39.771854 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870e1c0-45de-441a-8661-0122f8f80971 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:39.772161 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870e1c0-45de-441a-8661-0122f8f80971 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:39.773510 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870e1c0-45de-441a-8661-0122f8f80971 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:39.784529 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 262/512] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:39.785366 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870e1c0-45de-441a-8661-0122f8f80971 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:39.800591 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:39.824164 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:39.847280 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e502ff-c36d-4ba0-9c4b-c2d566ab53e4 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:39.879573 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388da6b8-6909-4b65-9851-d0adb12f1c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:39.880170 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388da6b8-6909-4b65-9851-d0adb12f1c60 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:39.880455 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388da6b8-6909-4b65-9851-d0adb12f1c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:39.880730 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388da6b8-6909-4b65-9851-d0adb12f1c60 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:39.881472 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388da6b8-6909-4b65-9851-d0adb12f1c60 None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:39.881783 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388da6b8-6909-4b65-9851-d0adb12f1c60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:39.883144 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388da6b8-6909-4b65-9851-d0adb12f1c60 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:39.883727 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870e1c0-45de-441a-8661-0122f8f80971 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:39.890840 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 251/513] 10.222.0.41 () {56 vars in 1103 bytes} [Fri Jun 3 16:02:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:39.891626 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388da6b8-6909-4b65-9851-d0adb12f1c60 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:39.919918 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3d2d4-6f91-42dc-8e32-5716b26ef16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:39.920575 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3d2d4-6f91-42dc-8e32-5716b26ef16c None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:39.920860 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3d2d4-6f91-42dc-8e32-5716b26ef16c None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:39.921138 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3d2d4-6f91-42dc-8e32-5716b26ef16c None admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:39.921961 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f3d2d4-6f91-42dc-8e32-5716b26ef16c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:39.922270 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f3d2d4-6f91-42dc-8e32-5716b26ef16c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:39.923702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f3d2d4-6f91-42dc-8e32-5716b26ef16c None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:39.959237 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 263/514] 10.222.0.41 () {56 vars in 1137 bytes} [Fri Jun 3 16:02:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:39.960049 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3d2d4-6f91-42dc-8e32-5716b26ef16c None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:39.989301 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388da6b8-6909-4b65-9851-d0adb12f1c60 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:40.021712 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dec0de-bfea-43b1-b3ca-19ea5de8e44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:40.022375 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dec0de-bfea-43b1-b3ca-19ea5de8e44a None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:40.022640 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dec0de-bfea-43b1-b3ca-19ea5de8e44a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:40.022914 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dec0de-bfea-43b1-b3ca-19ea5de8e44a None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:40.023645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dec0de-bfea-43b1-b3ca-19ea5de8e44a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:40.025099 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dec0de-bfea-43b1-b3ca-19ea5de8e44a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:40.030196 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:40.036279 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:40.047056 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 252/515] 10.222.0.41 () {60 vars in 1132 bytes} [Fri Jun 3 16:02:39 2022] POST /identity/v3/users => generated 247 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:02:40.047928 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dec0de-bfea-43b1-b3ca-19ea5de8e44a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:40.064574 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3d2d4-6f91-42dc-8e32-5716b26ef16c None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:40.105760 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0635e7a-bf6c-4afc-961b-7484206a5fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:40.106456 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0635e7a-bf6c-4afc-961b-7484206a5fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:40.106745 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0635e7a-bf6c-4afc-961b-7484206a5fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:40.107023 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0635e7a-bf6c-4afc-961b-7484206a5fd5 None admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:40.107925 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0635e7a-bf6c-4afc-961b-7484206a5fd5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:40.108241 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0635e7a-bf6c-4afc-961b-7484206a5fd5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:40.109720 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0635e7a-bf6c-4afc-961b-7484206a5fd5 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:40.133547 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 264/516] 10.222.0.41 () {56 vars in 1137 bytes} [Fri Jun 3 16:02:39 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:40.133547 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0635e7a-bf6c-4afc-961b-7484206a5fd5 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:40.172073 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dec0de-bfea-43b1-b3ca-19ea5de8e44a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:40.211574 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e647f5d1-af70-4cc5-97d2-6165879b4b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:40.212338 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647f5d1-af70-4cc5-97d2-6165879b4b78 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:40.212627 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647f5d1-af70-4cc5-97d2-6165879b4b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:40.212906 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647f5d1-af70-4cc5-97d2-6165879b4b78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:40.220966 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e647f5d1-af70-4cc5-97d2-6165879b4b78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:40.222748 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e647f5d1-af70-4cc5-97d2-6165879b4b78 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:40.240282 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0635e7a-bf6c-4afc-961b-7484206a5fd5 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:40.242919 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-e647f5d1-af70-4cc5-97d2-6165879b4b78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:02:40.242919 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:40.242919 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:40.242919 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:40.242919 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:40.242919 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:40.242919 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:40.242919 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:40.242919 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:40.242919 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:40.242919 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:40.242919 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:40.242919 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:02:40.242919 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:02:40.242919 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:02:40.243792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:02:40.243792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:40.243792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:40.243792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:40.243792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:40.243792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:40.243792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:40.243792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:40.243792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:40.243792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:40.243792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:40.243792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:40.243792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:40.243792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:40.243792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:40.243792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:40.243792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:40.244630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:02:40.244630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:02:40.244630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:02:40.244630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:02:40.244630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:02:40.244630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:02:40.244630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:02:40.244630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:40.244630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 253/517] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:02:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:40.245116 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e647f5d1-af70-4cc5-97d2-6165879b4b78 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:40.277693 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:40.287669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee9c062-7345-4f87-93aa-57f38adfdec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:40.288586 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9c062-7345-4f87-93aa-57f38adfdec9 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:40.288876 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9c062-7345-4f87-93aa-57f38adfdec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:40.289937 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9c062-7345-4f87-93aa-57f38adfdec9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:40.302210 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee9c062-7345-4f87-93aa-57f38adfdec9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:40.303690 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee9c062-7345-4f87-93aa-57f38adfdec9 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:40.312906 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:40.323702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-8ee9c062-7345-4f87-93aa-57f38adfdec9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:40.323702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:40.323702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:40.323702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:40.323702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:40.323702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:40.323702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:40.323702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:40.323702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:40.323702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:40.323702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:40.323702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:40.323702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:40.323702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:40.323702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:40.324560 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:40.324560 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:40.324560 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:40.324560 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:40.324560 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:40.324560 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:40.324560 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:40.324560 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:40.324560 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:40.324560 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:40.324560 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:40.324560 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:40.324560 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:40.324560 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:40.324560 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:40.324560 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:40.324560 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:40.325429 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:40.325429 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:40.325429 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:40.325429 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:40.325429 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 265/518] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:40.326196 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee9c062-7345-4f87-93aa-57f38adfdec9 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:40.367949 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e647f5d1-af70-4cc5-97d2-6165879b4b78 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:40.409690 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65d036-5198-4c42-8383-6bf1329e412b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:40.410852 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65d036-5198-4c42-8383-6bf1329e412b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:40.411474 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65d036-5198-4c42-8383-6bf1329e412b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:40.411756 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65d036-5198-4c42-8383-6bf1329e412b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:40.432664 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee9c062-7345-4f87-93aa-57f38adfdec9 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:40.468877 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32766b02-3eee-4d49-97cc-6d41207b2556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:40.469525 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32766b02-3eee-4d49-97cc-6d41207b2556 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:40.469808 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32766b02-3eee-4d49-97cc-6d41207b2556 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:40.470081 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32766b02-3eee-4d49-97cc-6d41207b2556 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:40.483966 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d65d036-5198-4c42-8383-6bf1329e412b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:40.485414 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d65d036-5198-4c42-8383-6bf1329e412b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:40.524697 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:40.541912 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32766b02-3eee-4d49-97cc-6d41207b2556 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:40.543375 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32766b02-3eee-4d49-97cc-6d41207b2556 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:40.544235 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:40.591894 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 254/519] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:02:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 16:02:40.592735 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65d036-5198-4c42-8383-6bf1329e412b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:40.654334 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 266/520] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:02:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 16:02:40.655576 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32766b02-3eee-4d49-97cc-6d41207b2556 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:40.702505 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65d036-5198-4c42-8383-6bf1329e412b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:40.739917 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98eef9-7a19-4afe-a1ac-64908284766d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:40.740656 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98eef9-7a19-4afe-a1ac-64908284766d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:40.740946 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98eef9-7a19-4afe-a1ac-64908284766d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:40.741224 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98eef9-7a19-4afe-a1ac-64908284766d None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:40.742009 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da98eef9-7a19-4afe-a1ac-64908284766d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:40.742139 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:40.742317 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da98eef9-7a19-4afe-a1ac-64908284766d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:40.743833 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da98eef9-7a19-4afe-a1ac-64908284766d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:40.762025 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32766b02-3eee-4d49-97cc-6d41207b2556 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:40.767203 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 255/521] 10.222.0.41 () {56 vars in 1148 bytes} [Fri Jun 3 16:02:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:40.767203 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98eef9-7a19-4afe-a1ac-64908284766d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:40.783600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:40.802925 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a643b8-6e93-492f-a56b-6f0a4e989aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:40.803730 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a643b8-6e93-492f-a56b-6f0a4e989aeb None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:40.803913 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a643b8-6e93-492f-a56b-6f0a4e989aeb None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:40.804198 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a643b8-6e93-492f-a56b-6f0a4e989aeb None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:40.804922 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a643b8-6e93-492f-a56b-6f0a4e989aeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:40.805233 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a643b8-6e93-492f-a56b-6f0a4e989aeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:40.806599 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a643b8-6e93-492f-a56b-6f0a4e989aeb None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:40.823817 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 267/522] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:40.823817 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a643b8-6e93-492f-a56b-6f0a4e989aeb None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:40.877247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98eef9-7a19-4afe-a1ac-64908284766d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:40.918854 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e6509c-e02a-4902-ae44-07065991d55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:40.920484 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6509c-e02a-4902-ae44-07065991d55e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:40.921234 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6509c-e02a-4902-ae44-07065991d55e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:40.921729 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6509c-e02a-4902-ae44-07065991d55e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:40.931589 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a643b8-6e93-492f-a56b-6f0a4e989aeb None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:40.936151 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e6509c-e02a-4902-ae44-07065991d55e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:40.937815 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e6509c-e02a-4902-ae44-07065991d55e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:40.958731 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-c6e6509c-e02a-4902-ae44-07065991d55e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:40.958731 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:40.958731 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:40.958731 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:40.958731 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:40.958731 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:40.958731 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:40.958731 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:40.958731 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:40.958731 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:40.958731 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:40.958731 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:40.958731 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:40.958731 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:40.958731 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:40.964903 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:40.964903 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:40.964903 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:40.964903 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:40.964903 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:40.964903 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:40.964903 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:40.964903 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:40.964903 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:40.964903 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:40.964903 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:40.964903 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:40.964903 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:40.964903 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:40.964903 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:40.964903 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:40.964903 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:40.965740 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:40.965740 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:40.965740 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:40.965740 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:40.965740 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 256/523] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:40.965740 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e6509c-e02a-4902-ae44-07065991d55e None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:40.973643 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f361ca-fdd4-4c03-a0d8-683cb40929f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:40.974442 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f361ca-fdd4-4c03-a0d8-683cb40929f4 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:40.974731 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f361ca-fdd4-4c03-a0d8-683cb40929f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:40.975012 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f361ca-fdd4-4c03-a0d8-683cb40929f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:40.987123 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f361ca-fdd4-4c03-a0d8-683cb40929f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:40.989108 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f361ca-fdd4-4c03-a0d8-683cb40929f4 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:40.991351 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:41.009015 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-72f361ca-fdd4-4c03-a0d8-683cb40929f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:41.009015 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:41.009015 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:41.009015 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:41.009015 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:41.009015 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:41.009015 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:41.009015 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:41.009015 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:41.009015 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:41.009015 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:41.009015 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:41.009015 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:41.009015 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:41.009015 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:41.009901 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:41.009901 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:41.009901 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:41.009901 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:41.009901 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:41.009901 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:41.009901 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:41.009901 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:41.009901 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:41.009901 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:41.009901 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:41.009901 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:41.009901 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:41.009901 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:41.009901 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:41.009901 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:41.009901 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:41.010837 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:41.010837 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:41.010837 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:41.010837 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:41.010837 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 268/524] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:41.011111 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f361ca-fdd4-4c03-a0d8-683cb40929f4 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:41.040768 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:41.079058 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e6509c-e02a-4902-ae44-07065991d55e None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:41.117993 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282dbc4c-3b04-49f2-b8a2-b60b69e2876f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:41.118649 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282dbc4c-3b04-49f2-b8a2-b60b69e2876f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:41.118947 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282dbc4c-3b04-49f2-b8a2-b60b69e2876f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:41.119227 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282dbc4c-3b04-49f2-b8a2-b60b69e2876f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:41.120099 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282dbc4c-3b04-49f2-b8a2-b60b69e2876f None admin] RBAC: Adding query filter params (role.id=fe695aedbe8249a28c992a19264bc34b, scope.project.id=c274254e510a468e9acbecc6298301cc, user.id=2fd08ea39ee94b92ac6763608e69f158) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:41.120417 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282dbc4c-3b04-49f2-b8a2-b60b69e2876f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:41.121852 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282dbc4c-3b04-49f2-b8a2-b60b69e2876f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:41.122774 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f361ca-fdd4-4c03-a0d8-683cb40929f4 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:41.135429 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 257/525] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:02:40 2022] GET /identity/v3/role_assignments?user.id=2fd08ea39ee94b92ac6763608e69f158&scope.project.id=c274254e510a468e9acbecc6298301cc&role.id=fe695aedbe8249a28c992a19264bc34b => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:41.135429 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282dbc4c-3b04-49f2-b8a2-b60b69e2876f None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:41.178088 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8487b47-fe4e-4638-873d-41f2ef7d3588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:41.179105 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8487b47-fe4e-4638-873d-41f2ef7d3588 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:41.179663 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8487b47-fe4e-4638-873d-41f2ef7d3588 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:41.180046 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8487b47-fe4e-4638-873d-41f2ef7d3588 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:41.189354 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8487b47-fe4e-4638-873d-41f2ef7d3588 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:41.191218 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8487b47-fe4e-4638-873d-41f2ef7d3588 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:41.208201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-c8487b47-fe4e-4638-873d-41f2ef7d3588 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 03 16:02:41.208201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:41.208201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:41.208201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:41.208201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:41.208201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:41.208201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:41.208201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:41.208201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:41.208201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:41.208201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:41.208201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:41.208201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:02:41.208201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:02:41.208201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:02:41.208201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:02:41.209215 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:41.209215 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:41.209215 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:02:41.209215 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:02:41.209215 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:02:41.209215 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:02:41.209215 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:41.209215 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:41.209215 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:41.209215 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:41.209215 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:41.209215 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:41.209215 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:41.209215 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:41.209215 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:41.209215 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:41.209215 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:41.209215 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:41.210192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:41.210192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:41.210192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:02:41.210192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:02:41.210192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:02:41.210192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:02:41.210192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:02:41.210192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:02:41.210192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 03 16:02:41.210192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:41.214822 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 269/526] 10.222.0.41 () {56 vars in 1102 bytes} [Fri Jun 3 16:02:41 2022] GET /identity/v3/users/neutron => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:41.215967 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8487b47-fe4e-4638-873d-41f2ef7d3588 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:41.279446 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282dbc4c-3b04-49f2-b8a2-b60b69e2876f None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:41.287872 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:41.318995 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:41.327420 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085e426-a95a-4be6-9074-911c3a4dfe59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:41.328101 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085e426-a95a-4be6-9074-911c3a4dfe59 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:41.328653 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085e426-a95a-4be6-9074-911c3a4dfe59 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:41.328936 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085e426-a95a-4be6-9074-911c3a4dfe59 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:41.330110 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085e426-a95a-4be6-9074-911c3a4dfe59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:41.330431 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085e426-a95a-4be6-9074-911c3a4dfe59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:41.331953 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085e426-a95a-4be6-9074-911c3a4dfe59 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:41.347989 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8487b47-fe4e-4638-873d-41f2ef7d3588 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:41.351423 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 258/527] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:41.353058 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085e426-a95a-4be6-9074-911c3a4dfe59 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:41.395619 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69bbb05-0c3b-4898-b979-a44e2c3d4112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:41.396281 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bbb05-0c3b-4898-b979-a44e2c3d4112 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:41.396568 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bbb05-0c3b-4898-b979-a44e2c3d4112 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:41.396843 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bbb05-0c3b-4898-b979-a44e2c3d4112 None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:41.397688 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69bbb05-0c3b-4898-b979-a44e2c3d4112 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:41.398000 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69bbb05-0c3b-4898-b979-a44e2c3d4112 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:41.399507 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69bbb05-0c3b-4898-b979-a44e2c3d4112 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:41.413118 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 270/528] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:41.414057 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69bbb05-0c3b-4898-b979-a44e2c3d4112 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:41.488843 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085e426-a95a-4be6-9074-911c3a4dfe59 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:41.520826 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69bbb05-0c3b-4898-b979-a44e2c3d4112 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:41.534189 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95b6e9-d382-4603-b010-8d3c207b62e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:41.535146 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95b6e9-d382-4603-b010-8d3c207b62e2 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:41.535929 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:41.536892 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95b6e9-d382-4603-b010-8d3c207b62e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:41.537579 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95b6e9-d382-4603-b010-8d3c207b62e2 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:41.538439 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95b6e9-d382-4603-b010-8d3c207b62e2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:41.539285 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95b6e9-d382-4603-b010-8d3c207b62e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:41.541436 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95b6e9-d382-4603-b010-8d3c207b62e2 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:41.557985 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d152250-1653-42b1-8026-6f048c409f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:41.558885 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d152250-1653-42b1-8026-6f048c409f7d None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:41.559240 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d152250-1653-42b1-8026-6f048c409f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:41.559628 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d152250-1653-42b1-8026-6f048c409f7d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:41.564829 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 259/529] 10.222.0.41 () {56 vars in 1139 bytes} [Fri Jun 3 16:02:41 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:41.565732 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95b6e9-d382-4603-b010-8d3c207b62e2 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:41.571258 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d152250-1653-42b1-8026-6f048c409f7d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:41.573106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d152250-1653-42b1-8026-6f048c409f7d None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:41.588539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-3d152250-1653-42b1-8026-6f048c409f7d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:02:41.588539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:41.588539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:41.588539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:41.588539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:41.588539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:41.588539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:41.588539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:41.588539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:41.588539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:41.588539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:41.588539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:41.588539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:02:41.588539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:02:41.588539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:02:41.589403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:02:41.589403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:41.589403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:41.589403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:41.589403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:41.589403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:41.589403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:41.589403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:41.589403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:41.589403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:41.589403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:41.589403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:41.589403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:41.589403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:41.589403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:41.589403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:41.589403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:41.590231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:02:41.590231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:02:41.590231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:02:41.590231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:02:41.590231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:02:41.590231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:02:41.590231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:02:41.590231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:41.590231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 271/530] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:02:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:41.590679 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d152250-1653-42b1-8026-6f048c409f7d None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:41.637223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:41.696352 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95b6e9-d382-4603-b010-8d3c207b62e2 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:41.730077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d152250-1653-42b1-8026-6f048c409f7d None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:41.735002 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5cd51b-b590-47d9-948e-36cfe02e3ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:41.735836 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5cd51b-b590-47d9-948e-36cfe02e3ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:41.736124 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5cd51b-b590-47d9-948e-36cfe02e3ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:41.736404 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5cd51b-b590-47d9-948e-36cfe02e3ad2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:41.745283 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5cd51b-b590-47d9-948e-36cfe02e3ad2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:41.746709 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5cd51b-b590-47d9-948e-36cfe02e3ad2 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:41.761067 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-2f5cd51b-b590-47d9-948e-36cfe02e3ad2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:02:41.761067 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:41.761067 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:41.761067 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:41.761067 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:41.761067 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:41.761067 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:41.761067 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:41.761067 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:41.761067 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:41.761067 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:41.761067 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:41.761067 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:02:41.761067 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:02:41.761067 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:02:41.761921 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:02:41.761921 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:41.761921 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:41.761921 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:41.761921 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:41.761921 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:41.761921 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:41.761921 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:41.761921 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:41.761921 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:41.761921 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:41.761921 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:41.761921 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:41.761921 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:41.761921 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:41.761921 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:41.761921 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:41.762784 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:02:41.762784 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:02:41.762784 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:02:41.762784 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:02:41.762784 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:02:41.762784 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:02:41.762784 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:02:41.762784 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:41.762784 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 260/531] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:02:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:41.763712 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5cd51b-b590-47d9-948e-36cfe02e3ad2 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:41.774896 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446416fd-271c-457f-bef6-1cfcf7e21829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:41.775600 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446416fd-271c-457f-bef6-1cfcf7e21829 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:41.776335 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446416fd-271c-457f-bef6-1cfcf7e21829 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:41.776622 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446416fd-271c-457f-bef6-1cfcf7e21829 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:41.839734 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:41.863495 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446416fd-271c-457f-bef6-1cfcf7e21829 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:41.864987 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446416fd-271c-457f-bef6-1cfcf7e21829 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:41.884981 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5cd51b-b590-47d9-948e-36cfe02e3ad2 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:41.900406 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:41.926027 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16124d-2269-4fab-b8d8-8adaacb90266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:41.926697 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16124d-2269-4fab-b8d8-8adaacb90266 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:41.926984 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16124d-2269-4fab-b8d8-8adaacb90266 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:41.927260 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16124d-2269-4fab-b8d8-8adaacb90266 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:41.928079 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce16124d-2269-4fab-b8d8-8adaacb90266 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:41.928389 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce16124d-2269-4fab-b8d8-8adaacb90266 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:41.929820 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce16124d-2269-4fab-b8d8-8adaacb90266 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:41.943914 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 261/532] 10.222.0.41 () {56 vars in 1148 bytes} [Fri Jun 3 16:02:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:41.944762 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16124d-2269-4fab-b8d8-8adaacb90266 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:42.004138 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 272/533] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:02:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 16:02:42.005354 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446416fd-271c-457f-bef6-1cfcf7e21829 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:42.059143 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16124d-2269-4fab-b8d8-8adaacb90266 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:42.100309 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d0687-a507-44bc-8dad-a30bcd809122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:42.101570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d0687-a507-44bc-8dad-a30bcd809122 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:42.102906 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d0687-a507-44bc-8dad-a30bcd809122 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:42.103451 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d0687-a507-44bc-8dad-a30bcd809122 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:42.104434 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385d0687-a507-44bc-8dad-a30bcd809122 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:42.104937 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385d0687-a507-44bc-8dad-a30bcd809122 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:42.106577 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385d0687-a507-44bc-8dad-a30bcd809122 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:42.107499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:42.128182 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 262/534] 10.222.0.41 () {56 vars in 1148 bytes} [Fri Jun 3 16:02:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:42.128182 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d0687-a507-44bc-8dad-a30bcd809122 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:42.139753 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446416fd-271c-457f-bef6-1cfcf7e21829 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:42.168016 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:42.176900 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c555e-aba6-4902-bd6f-674ed45f6364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:42.177673 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c555e-aba6-4902-bd6f-674ed45f6364 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:42.178059 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c555e-aba6-4902-bd6f-674ed45f6364 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:42.178454 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c555e-aba6-4902-bd6f-674ed45f6364 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:42.179417 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c555e-aba6-4902-bd6f-674ed45f6364 None admin] RBAC: Adding query filter params (role.id=fe695aedbe8249a28c992a19264bc34b, scope.project.id=c274254e510a468e9acbecc6298301cc, user.id=62a3bbb9720049b29eb37b0dd88bbad7) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:42.179834 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c555e-aba6-4902-bd6f-674ed45f6364 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:42.181632 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c555e-aba6-4902-bd6f-674ed45f6364 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:42.187430 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 273/535] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:02:42 2022] GET /identity/v3/role_assignments?user.id=62a3bbb9720049b29eb37b0dd88bbad7&scope.project.id=c274254e510a468e9acbecc6298301cc&role.id=fe695aedbe8249a28c992a19264bc34b => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:42.188307 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c555e-aba6-4902-bd6f-674ed45f6364 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:42.265329 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d0687-a507-44bc-8dad-a30bcd809122 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:42.317397 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7debe-e041-469a-b59c-d15785d6a4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:42.318265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7debe-e041-469a-b59c-d15785d6a4a0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:42.320202 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7debe-e041-469a-b59c-d15785d6a4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:42.320502 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7debe-e041-469a-b59c-d15785d6a4a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:42.323192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c555e-aba6-4902-bd6f-674ed45f6364 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:42.345332 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a7debe-e041-469a-b59c-d15785d6a4a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:42.346813 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a7debe-e041-469a-b59c-d15785d6a4a0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:42.381489 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-d7a7debe-e041-469a-b59c-d15785d6a4a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:42.381489 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:42.381489 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:42.381489 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:42.381489 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:42.381489 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:42.381489 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:42.381489 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:42.381489 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:42.381489 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:42.381489 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:42.381489 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:42.381489 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:42.381489 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:42.381489 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:42.382393 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:42.382393 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:42.382393 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:42.382393 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:42.382393 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:42.382393 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:42.382393 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:42.382393 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:42.382393 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:42.382393 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:42.382393 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:42.382393 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:42.382393 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:42.382393 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:42.382393 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:42.382393 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:42.382393 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:42.383335 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:42.383335 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:42.383335 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:42.383335 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:42.383335 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 263/536] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:42.388583 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7debe-e041-469a-b59c-d15785d6a4a0 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:42.412368 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f360d2b0-44e5-4f52-8c81-4c2112cb298c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:42.413052 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360d2b0-44e5-4f52-8c81-4c2112cb298c None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:42.413335 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360d2b0-44e5-4f52-8c81-4c2112cb298c None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:42.413874 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360d2b0-44e5-4f52-8c81-4c2112cb298c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:42.416290 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f360d2b0-44e5-4f52-8c81-4c2112cb298c None admin] RBAC: Adding query filter params (role.id=fe695aedbe8249a28c992a19264bc34b, scope.project.id=c274254e510a468e9acbecc6298301cc, user.id=bbaf66d6e1f54047917cc615546bbd29) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:42.416616 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f360d2b0-44e5-4f52-8c81-4c2112cb298c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:42.418083 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f360d2b0-44e5-4f52-8c81-4c2112cb298c None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:42.432760 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 274/537] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:02:42 2022] GET /identity/v3/role_assignments?user.id=bbaf66d6e1f54047917cc615546bbd29&scope.project.id=c274254e510a468e9acbecc6298301cc&role.id=fe695aedbe8249a28c992a19264bc34b => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:42.473698 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091aaff-5f80-4a73-a8fa-74daaf0494b8 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:42.473991 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091aaff-5f80-4a73-a8fa-74daaf0494b8 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:42.474433 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091aaff-5f80-4a73-a8fa-74daaf0494b8 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:42.476479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 275/538] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 16:02:42.483369 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006770d5-d8f1-4ac5-8a22-1bf7bf7b71ee None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:42.483804 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006770d5-d8f1-4ac5-8a22-1bf7bf7b71ee None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:42.484208 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006770d5-d8f1-4ac5-8a22-1bf7bf7b71ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:42.492095 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:42.508215 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-006770d5-d8f1-4ac5-8a22-1bf7bf7b71ee None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:42.509780 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-006770d5-d8f1-4ac5-8a22-1bf7bf7b71ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:42.514241 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:42.535493 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7debe-e041-469a-b59c-d15785d6a4a0 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:42.572457 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca294b7-fc20-46c8-9f36-d6ec390eb68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:42.573381 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca294b7-fc20-46c8-9f36-d6ec390eb68c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:42.573808 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca294b7-fc20-46c8-9f36-d6ec390eb68c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:42.574248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca294b7-fc20-46c8-9f36-d6ec390eb68c None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:42.575166 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca294b7-fc20-46c8-9f36-d6ec390eb68c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:42.575686 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca294b7-fc20-46c8-9f36-d6ec390eb68c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:42.577296 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca294b7-fc20-46c8-9f36-d6ec390eb68c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:42.590480 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 264/539] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:02:42.596100 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca294b7-fc20-46c8-9f36-d6ec390eb68c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:42.598456 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 276/540] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:42.622843 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006770d5-d8f1-4ac5-8a22-1bf7bf7b71ee None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:42.757323 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006770d5-d8f1-4ac5-8a22-1bf7bf7b71ee None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:42.812810 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca294b7-fc20-46c8-9f36-d6ec390eb68c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:42.817060 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:42.828759 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555cedd4-1496-42fe-be7f-ccaac583d0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:42.829566 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555cedd4-1496-42fe-be7f-ccaac583d0f2 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:42.829852 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555cedd4-1496-42fe-be7f-ccaac583d0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:42.830132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555cedd4-1496-42fe-be7f-ccaac583d0f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:42.854692 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555cedd4-1496-42fe-be7f-ccaac583d0f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:42.862643 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-555cedd4-1496-42fe-be7f-ccaac583d0f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:02:42.862643 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:42.862643 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:42.862643 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:42.862643 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:42.862643 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:42.862643 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:42.862643 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:42.862643 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:42.862643 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:42.862643 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:42.862643 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:42.862643 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:42.862643 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:42.862643 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:42.862643 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:42.863595 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:42.863595 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:42.863595 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:42.863595 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:42.863595 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:42.863595 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:42.863595 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:42.863595 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:42.863595 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:42.863595 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:42.863595 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:42.863595 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:42.863595 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:42.863595 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:42.863595 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:42.863595 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:42.864391 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:42.864391 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:42.864391 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:42.864391 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:42.864391 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:42.864391 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:42.864391 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:42.864391 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:42.864391 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:42.864391 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:42.864391 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:02:42.864391 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:42.865948 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 277/541] 10.222.0.41 () {56 vars in 1102 bytes} [Fri Jun 3 16:02:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:02:42.872418 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555cedd4-1496-42fe-be7f-ccaac583d0f2 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:42.874444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b59ce0-f690-4bdd-99e8-e220a68476a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:42.875242 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b59ce0-f690-4bdd-99e8-e220a68476a5 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:42.875552 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b59ce0-f690-4bdd-99e8-e220a68476a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:42.875829 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b59ce0-f690-4bdd-99e8-e220a68476a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:42.883579 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b59ce0-f690-4bdd-99e8-e220a68476a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:42.885479 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b59ce0-f690-4bdd-99e8-e220a68476a5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:42.900310 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-c1b59ce0-f690-4bdd-99e8-e220a68476a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:02:42.900310 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:42.900310 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:42.900310 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:42.900310 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:42.900310 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:42.900310 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:42.900310 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:42.900310 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:42.900310 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:42.900310 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:42.900310 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:42.900310 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:02:42.900310 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:02:42.900310 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:02:42.901177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:02:42.901177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:42.901177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:42.901177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:42.901177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:42.901177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:42.901177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:42.901177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:42.901177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:42.901177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:42.901177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:42.901177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:42.901177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:42.901177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:42.901177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:42.901177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:42.901177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:42.902004 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:02:42.902004 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:02:42.902004 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:02:42.902004 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:02:42.902004 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:02:42.902004 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:02:42.902004 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:02:42.902004 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:42.902004 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 265/542] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:02:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:02:42.912248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b59ce0-f690-4bdd-99e8-e220a68476a5 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:42.915097 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:43.015750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555cedd4-1496-42fe-be7f-ccaac583d0f2 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:43.051029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b59ce0-f690-4bdd-99e8-e220a68476a5 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:43.072691 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36059430-adcb-4def-9298-e8845dc2ce10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:43.073596 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36059430-adcb-4def-9298-e8845dc2ce10 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:43.074037 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36059430-adcb-4def-9298-e8845dc2ce10 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:43.074505 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36059430-adcb-4def-9298-e8845dc2ce10 None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:43.075533 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36059430-adcb-4def-9298-e8845dc2ce10 None admin] RBAC: Adding query filter params (name=service) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:43.075981 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36059430-adcb-4def-9298-e8845dc2ce10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:43.077601 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36059430-adcb-4def-9298-e8845dc2ce10 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:43.085480 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec203281-847e-483e-8415-e156dbed707a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:43.086134 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec203281-847e-483e-8415-e156dbed707a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:43.086417 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec203281-847e-483e-8415-e156dbed707a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:43.086688 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec203281-847e-483e-8415-e156dbed707a None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:43.090460 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec203281-847e-483e-8415-e156dbed707a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:43.090460 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec203281-847e-483e-8415-e156dbed707a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:43.090460 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec203281-847e-483e-8415-e156dbed707a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:43.094850 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 278/543] 10.222.0.41 () {56 vars in 1103 bytes} [Fri Jun 3 16:02:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:02:43.100261 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36059430-adcb-4def-9298-e8845dc2ce10 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:43.102099 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 266/544] 10.222.0.41 () {56 vars in 1148 bytes} [Fri Jun 3 16:02:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:43.108708 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec203281-847e-483e-8415-e156dbed707a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:43.179977 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:43.214169 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec203281-847e-483e-8415-e156dbed707a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:43.245106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:43.251508 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ffabcf-46f7-4fc5-bc21-bd01bbb915fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:43.251810 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ffabcf-46f7-4fc5-bc21-bd01bbb915fd None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:43.252101 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ffabcf-46f7-4fc5-bc21-bd01bbb915fd None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:43.253381 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ffabcf-46f7-4fc5-bc21-bd01bbb915fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:43.254853 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ffabcf-46f7-4fc5-bc21-bd01bbb915fd None admin] RBAC: Adding query filter params (role.id=fe695aedbe8249a28c992a19264bc34b, scope.project.id=c274254e510a468e9acbecc6298301cc, user.id=aeeaad0680074179b3d29abcf0d51515) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:43.255478 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ffabcf-46f7-4fc5-bc21-bd01bbb915fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:43.258223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ffabcf-46f7-4fc5-bc21-bd01bbb915fd None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:43.270880 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 267/545] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:02:43 2022] GET /identity/v3/role_assignments?user.id=aeeaad0680074179b3d29abcf0d51515&scope.project.id=c274254e510a468e9acbecc6298301cc&role.id=fe695aedbe8249a28c992a19264bc34b => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:02:43.326273 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36059430-adcb-4def-9298-e8845dc2ce10 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:43.381305 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f4cc2-0420-48a6-b268-fb920f6f8675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:43.382105 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f4cc2-0420-48a6-b268-fb920f6f8675 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:43.382384 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f4cc2-0420-48a6-b268-fb920f6f8675 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:43.382655 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f4cc2-0420-48a6-b268-fb920f6f8675 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:43.400774 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4f4cc2-0420-48a6-b268-fb920f6f8675 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:43.402226 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4f4cc2-0420-48a6-b268-fb920f6f8675 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:43.423875 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-9b4f4cc2-0420-48a6-b268-fb920f6f8675 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:43.423875 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:43.423875 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:43.423875 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:43.423875 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:43.423875 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:43.423875 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:43.423875 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:43.423875 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:43.423875 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:43.423875 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:43.423875 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:43.423875 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:43.423875 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:43.423875 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:43.424783 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:43.424783 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:43.424783 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:43.424783 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:43.424783 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:43.424783 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:43.424783 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:43.424783 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:43.424783 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:43.424783 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:43.424783 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:43.424783 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:43.424783 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:43.424783 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:43.424783 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:43.424783 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:43.424783 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:43.426180 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:43.426180 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:43.426180 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:43.426180 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:43.426180 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 279/546] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:43.431460 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ffabcf-46f7-4fc5-bc21-bd01bbb915fd None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:43.544131 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ffabcf-46f7-4fc5-bc21-bd01bbb915fd None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:43.572149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:43.582838 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77bbd7-1794-404d-833d-cdf11cc968b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:43.583515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77bbd7-1794-404d-833d-cdf11cc968b9 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:43.583809 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77bbd7-1794-404d-833d-cdf11cc968b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:43.584097 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77bbd7-1794-404d-833d-cdf11cc968b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:43.584830 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f77bbd7-1794-404d-833d-cdf11cc968b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:43.585139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f77bbd7-1794-404d-833d-cdf11cc968b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:43.586520 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f77bbd7-1794-404d-833d-cdf11cc968b9 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:43.599986 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 268/547] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:43.604913 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f4cc2-0420-48a6-b268-fb920f6f8675 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:43.766118 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f4cc2-0420-48a6-b268-fb920f6f8675 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:43.773834 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f769c63-e1ea-4e7d-ac9e-81c30d6eb336 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:43.774831 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f769c63-e1ea-4e7d-ac9e-81c30d6eb336 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:43.775338 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f769c63-e1ea-4e7d-ac9e-81c30d6eb336 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:43.777837 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 269/548] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:43.779078 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:43.783356 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7609314-41f5-4fde-9d0d-1e59ce77eeb3 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:43.784340 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7609314-41f5-4fde-9d0d-1e59ce77eeb3 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:43.785098 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7609314-41f5-4fde-9d0d-1e59ce77eeb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:43.805861 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-e7609314-41f5-4fde-9d0d-1e59ce77eeb3 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:43.807607 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e7609314-41f5-4fde-9d0d-1e59ce77eeb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:43.827470 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff20383-717d-4853-b115-9fbbe19d3705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:43.828275 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff20383-717d-4853-b115-9fbbe19d3705 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:43.828558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff20383-717d-4853-b115-9fbbe19d3705 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:43.828833 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff20383-717d-4853-b115-9fbbe19d3705 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:43.839458 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff20383-717d-4853-b115-9fbbe19d3705 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:43.840989 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff20383-717d-4853-b115-9fbbe19d3705 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:43.863359 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-0ff20383-717d-4853-b115-9fbbe19d3705 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 03 16:02:43.863359 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:43.863359 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:43.863359 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:43.863359 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:43.863359 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:43.863359 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:43.863359 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:43.863359 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:43.863359 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:43.863359 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:43.863359 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:43.863359 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:02:43.863359 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:02:43.863359 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:02:43.863359 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:02:43.864779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:43.864779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:43.864779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:02:43.864779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:02:43.864779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:02:43.864779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:02:43.864779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:43.864779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:43.864779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:43.864779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:43.864779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:43.864779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:43.864779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:43.864779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:43.864779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:43.864779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:43.864779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:43.864779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:43.866262 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:43.866262 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:43.866262 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:02:43.866262 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:02:43.866262 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:02:43.866262 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:02:43.866262 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:02:43.866262 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:02:43.866262 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 03 16:02:43.866262 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:43.875550 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 280/549] 10.222.0.41 () {56 vars in 1096 bytes} [Fri Jun 3 16:02:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:43.876415 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff20383-717d-4853-b115-9fbbe19d3705 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:43.920403 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 270/550] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:43.973318 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7609314-41f5-4fde-9d0d-1e59ce77eeb3 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:44.078395 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff20383-717d-4853-b115-9fbbe19d3705 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:44.097104 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7609314-41f5-4fde-9d0d-1e59ce77eeb3 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:44.133076 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a22ef-9ec8-444c-af60-64243c57b1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:44.134098 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a22ef-9ec8-444c-af60-64243c57b1d3 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:44.134546 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a22ef-9ec8-444c-af60-64243c57b1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:44.134961 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a22ef-9ec8-444c-af60-64243c57b1d3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:44.146436 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a22ef-9ec8-444c-af60-64243c57b1d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:44.153356 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f8c05-0d10-456e-96aa-909f53e7699f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:44.154288 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f8c05-0d10-456e-96aa-909f53e7699f None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:44.154575 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f8c05-0d10-456e-96aa-909f53e7699f None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:44.154860 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f8c05-0d10-456e-96aa-909f53e7699f None admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:44.155671 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:44.155795 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9f8c05-0d10-456e-96aa-909f53e7699f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:44.156031 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9f8c05-0d10-456e-96aa-909f53e7699f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:44.157733 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9f8c05-0d10-456e-96aa-909f53e7699f None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:44.162178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-3d3a22ef-9ec8-444c-af60-64243c57b1d3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:02:44.162178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:44.162178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:44.162178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:44.162178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:44.162178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:44.162178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:44.162178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:44.162178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:44.162178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:44.162178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:44.162178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:44.162178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:44.162178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:44.162178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:44.162178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:44.163289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:44.163289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:44.163289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:44.163289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:44.163289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:44.163289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:44.163289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:44.163289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:44.163289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:44.163289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:44.163289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:44.163289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:44.163289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:44.163289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:44.163289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:44.163289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:44.164396 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:44.164396 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:44.164396 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:44.164396 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:44.164396 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:44.164396 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:44.164396 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:44.164396 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:44.164396 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:44.164396 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:44.164396 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:02:44.164396 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:44.165043 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 271/551] 10.222.0.41 () {56 vars in 1102 bytes} [Fri Jun 3 16:02:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:02:44.170664 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a22ef-9ec8-444c-af60-64243c57b1d3 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:44.181006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 281/552] 10.222.0.41 () {56 vars in 1135 bytes} [Fri Jun 3 16:02:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:44.186592 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f8c05-0d10-456e-96aa-909f53e7699f None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:44.212773 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:44.292351 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a22ef-9ec8-444c-af60-64243c57b1d3 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:44.300718 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f8c05-0d10-456e-96aa-909f53e7699f None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:44.327962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c4658-f65a-42ff-976e-af9926871385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:44.328978 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c4658-f65a-42ff-976e-af9926871385 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:44.329454 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c4658-f65a-42ff-976e-af9926871385 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:44.329911 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c4658-f65a-42ff-976e-af9926871385 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:44.330986 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1c4658-f65a-42ff-976e-af9926871385 None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:44.331537 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1c4658-f65a-42ff-976e-af9926871385 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:44.333204 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1c4658-f65a-42ff-976e-af9926871385 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:44.336709 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820b099-8dda-45e2-bb59-acfc66801b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:44.337548 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820b099-8dda-45e2-bb59-acfc66801b14 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:44.337978 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820b099-8dda-45e2-bb59-acfc66801b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:44.338395 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820b099-8dda-45e2-bb59-acfc66801b14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:44.342933 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 272/553] 10.222.0.41 () {56 vars in 1103 bytes} [Fri Jun 3 16:02:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:44.348647 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c4658-f65a-42ff-976e-af9926871385 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:44.409083 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820b099-8dda-45e2-bb59-acfc66801b14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:44.411097 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820b099-8dda-45e2-bb59-acfc66801b14 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:44.419812 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:44.436890 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:44.460452 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c4658-f65a-42ff-976e-af9926871385 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:44.516077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 282/554] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:02:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 16:02:44.522305 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820b099-8dda-45e2-bb59-acfc66801b14 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:44.541190 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2594cc23-48ea-4cfe-8de2-f6201f842bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:44.542226 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2594cc23-48ea-4cfe-8de2-f6201f842bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:44.542685 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2594cc23-48ea-4cfe-8de2-f6201f842bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:44.543104 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2594cc23-48ea-4cfe-8de2-f6201f842bf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:44.564888 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2594cc23-48ea-4cfe-8de2-f6201f842bf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:44.567140 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2594cc23-48ea-4cfe-8de2-f6201f842bf9 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:44.613598 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-2594cc23-48ea-4cfe-8de2-f6201f842bf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:44.613598 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:44.613598 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:44.613598 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:44.613598 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:44.613598 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:44.613598 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:44.613598 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:44.613598 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:44.613598 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:44.613598 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:44.613598 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:44.613598 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:44.613598 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:44.613598 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:44.614677 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:44.614677 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:44.614677 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:44.614677 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:44.614677 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:44.614677 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:44.614677 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:44.614677 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:44.614677 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:44.614677 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:44.614677 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:44.614677 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:44.614677 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:44.614677 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:44.614677 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:44.614677 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:44.614677 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:44.615775 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:44.615775 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:44.615775 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:44.615775 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:44.615775 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 273/555] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:02:44.615775 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2594cc23-48ea-4cfe-8de2-f6201f842bf9 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:44.668901 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820b099-8dda-45e2-bb59-acfc66801b14 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:44.703702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:44.710989 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b98580-6c0e-4bd0-abfa-545e818f6b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:44.713169 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b98580-6c0e-4bd0-abfa-545e818f6b8e None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:44.713610 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b98580-6c0e-4bd0-abfa-545e818f6b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:44.715564 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b98580-6c0e-4bd0-abfa-545e818f6b8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:44.745281 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b98580-6c0e-4bd0-abfa-545e818f6b8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:44.746737 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b98580-6c0e-4bd0-abfa-545e818f6b8e None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:44.792662 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:44.799726 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2594cc23-48ea-4cfe-8de2-f6201f842bf9 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:44.808179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-57b98580-6c0e-4bd0-abfa-545e818f6b8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:44.808179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:44.808179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:44.808179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:44.808179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:44.808179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:44.808179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:44.808179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:44.808179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:44.808179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:44.808179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:44.808179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:44.808179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:44.808179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:44.808179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:44.809183 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:44.809183 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:44.809183 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:44.809183 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:44.809183 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:44.809183 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:44.809183 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:44.809183 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:44.809183 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:44.809183 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:44.809183 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:44.809183 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:44.809183 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:44.809183 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:44.809183 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:44.809183 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:44.809183 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:44.810587 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:44.810587 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:44.810587 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:44.810587 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:44.810587 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 283/556] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:44.824306 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b98580-6c0e-4bd0-abfa-545e818f6b8e None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:44.845149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6669a09-0ef1-4678-a79b-3b7dc7d64cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:44.846194 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6669a09-0ef1-4678-a79b-3b7dc7d64cfd None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:44.847903 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6669a09-0ef1-4678-a79b-3b7dc7d64cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:44.848383 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6669a09-0ef1-4678-a79b-3b7dc7d64cfd None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:44.849338 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6669a09-0ef1-4678-a79b-3b7dc7d64cfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:44.849835 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6669a09-0ef1-4678-a79b-3b7dc7d64cfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:44.851889 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6669a09-0ef1-4678-a79b-3b7dc7d64cfd None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:44.867680 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 274/557] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:02:44.880475 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6669a09-0ef1-4678-a79b-3b7dc7d64cfd None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:45.043247 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b98580-6c0e-4bd0-abfa-545e818f6b8e None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:45.068633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6669a09-0ef1-4678-a79b-3b7dc7d64cfd None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:45.084077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d411de-f19b-4854-ab98-eb0a6224ae25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:45.084971 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d411de-f19b-4854-ab98-eb0a6224ae25 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:45.085407 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d411de-f19b-4854-ab98-eb0a6224ae25 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:45.085677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d411de-f19b-4854-ab98-eb0a6224ae25 None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:45.086544 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d411de-f19b-4854-ab98-eb0a6224ae25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:45.087016 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d411de-f19b-4854-ab98-eb0a6224ae25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:45.088593 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d411de-f19b-4854-ab98-eb0a6224ae25 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:45.102003 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 284/558] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:45.103750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc7143-d9df-4d19-8847-ff4a4ded2877 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:45.104216 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc7143-d9df-4d19-8847-ff4a4ded2877 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:45.104691 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc7143-d9df-4d19-8847-ff4a4ded2877 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:45.106311 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 285/559] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:45.107983 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc7143-d9df-4d19-8847-ff4a4ded2877 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:45.118493 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48011201-0cbf-4b6c-9b9e-2b89a6580c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:45.119224 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48011201-0cbf-4b6c-9b9e-2b89a6580c3b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:45.119559 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48011201-0cbf-4b6c-9b9e-2b89a6580c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:45.119836 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48011201-0cbf-4b6c-9b9e-2b89a6580c3b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:45.128665 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48011201-0cbf-4b6c-9b9e-2b89a6580c3b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:45.130336 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48011201-0cbf-4b6c-9b9e-2b89a6580c3b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:45.144876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-48011201-0cbf-4b6c-9b9e-2b89a6580c3b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 03 16:02:45.144876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:45.144876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:45.144876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:45.144876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:45.144876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:45.144876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:45.144876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:45.144876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:45.144876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:45.144876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:45.144876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:45.144876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:02:45.144876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:02:45.144876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:02:45.144876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:02:45.145814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:45.145814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:45.145814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:02:45.145814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:02:45.145814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:02:45.145814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:02:45.145814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:45.145814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:45.145814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:45.145814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:45.145814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:45.145814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:45.145814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:45.145814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:45.145814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:45.145814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:45.145814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:45.145814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:45.146732 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:45.146732 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:45.146732 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:02:45.146732 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:02:45.146732 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:02:45.146732 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:02:45.146732 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:02:45.146732 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:02:45.146732 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 03 16:02:45.146732 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:45.149383 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 275/560] 10.222.0.41 () {56 vars in 1093 bytes} [Fri Jun 3 16:02:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:45.155769 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268b50b-5ae5-4790-b6be-9a2be9b0c643 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:45.155769 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268b50b-5ae5-4790-b6be-9a2be9b0c643 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:45.155769 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268b50b-5ae5-4790-b6be-9a2be9b0c643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:45.160859 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:45.176968 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-f268b50b-5ae5-4790-b6be-9a2be9b0c643 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:45.178524 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f268b50b-5ae5-4790-b6be-9a2be9b0c643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:45.181900 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:45.238615 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc7143-d9df-4d19-8847-ff4a4ded2877 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:45.268887 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 276/561] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:45.270550 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268b50b-5ae5-4790-b6be-9a2be9b0c643 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:45.285791 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134cade-0ffe-4493-9b16-9a5998b846cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:45.286957 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134cade-0ffe-4493-9b16-9a5998b846cb None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:45.287534 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134cade-0ffe-4493-9b16-9a5998b846cb None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:45.288022 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134cade-0ffe-4493-9b16-9a5998b846cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:45.296419 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5134cade-0ffe-4493-9b16-9a5998b846cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:45.298087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5134cade-0ffe-4493-9b16-9a5998b846cb None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:45.312821 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-5134cade-0ffe-4493-9b16-9a5998b846cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:02:45.312821 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:45.312821 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:45.312821 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:45.312821 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:45.312821 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:45.312821 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:45.312821 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:45.312821 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:45.312821 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:45.312821 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:45.312821 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:45.312821 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:02:45.312821 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:02:45.312821 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:02:45.314131 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:02:45.314131 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:45.314131 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:45.314131 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:45.314131 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:45.314131 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:45.314131 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:45.314131 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:45.314131 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:45.314131 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:45.314131 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:45.314131 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:45.314131 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:45.314131 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:45.314131 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:45.314131 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:45.314131 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:45.315834 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:02:45.315834 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:02:45.315834 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:02:45.315834 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:02:45.315834 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:02:45.315834 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:02:45.315834 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:02:45.315834 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:45.315834 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 286/562] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:02:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:45.315834 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a6e573-d15d-441c-a62c-015a53014ecc None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:45.315834 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a6e573-d15d-441c-a62c-015a53014ecc None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:45.316645 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a6e573-d15d-441c-a62c-015a53014ecc None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:45.321022 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 287/563] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:45.321805 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a6e573-d15d-441c-a62c-015a53014ecc None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:45.387120 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268b50b-5ae5-4790-b6be-9a2be9b0c643 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:45.423135 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:45.425823 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc060c61-947f-4fa2-93d9-a82df792205b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:45.426543 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc060c61-947f-4fa2-93d9-a82df792205b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:45.426902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc060c61-947f-4fa2-93d9-a82df792205b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:45.427256 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc060c61-947f-4fa2-93d9-a82df792205b None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:45.428309 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc060c61-947f-4fa2-93d9-a82df792205b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:45.428693 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc060c61-947f-4fa2-93d9-a82df792205b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:45.430135 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc060c61-947f-4fa2-93d9-a82df792205b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:45.431690 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a6e573-d15d-441c-a62c-015a53014ecc None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:45.440693 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:45.450792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 277/564] 10.222.0.41 () {56 vars in 1133 bytes} [Fri Jun 3 16:02:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:45.452236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc060c61-947f-4fa2-93d9-a82df792205b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:45.473020 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9fcf56-c358-4916-b6c1-dfb25447d89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:45.473809 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9fcf56-c358-4916-b6c1-dfb25447d89e None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:45.474213 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9fcf56-c358-4916-b6c1-dfb25447d89e None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:45.474492 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9fcf56-c358-4916-b6c1-dfb25447d89e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:45.485877 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9fcf56-c358-4916-b6c1-dfb25447d89e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:45.494258 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-9c9fcf56-c358-4916-b6c1-dfb25447d89e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:02:45.494258 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:45.494258 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:45.494258 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:45.494258 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:45.494258 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:45.494258 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:45.494258 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:45.494258 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:45.494258 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:45.494258 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:45.494258 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:45.494258 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:45.494258 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:45.494258 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:45.494258 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:45.495180 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:45.495180 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:45.495180 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:45.495180 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:45.495180 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:45.495180 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:45.495180 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:45.495180 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:45.495180 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:45.495180 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:45.495180 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:45.495180 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:45.495180 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:45.495180 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:45.495180 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:45.495180 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:45.500986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:45.500986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:45.500986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:45.500986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:45.500986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:45.500986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:45.500986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:45.500986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:45.500986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:45.500986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:45.500986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:02:45.500986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:45.500986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 288/565] 10.222.0.41 () {56 vars in 1102 bytes} [Fri Jun 3 16:02:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:45.500986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4b211-c549-4252-852b-db48d052e2f9 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:45.501896 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4b211-c549-4252-852b-db48d052e2f9 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:45.501896 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4b211-c549-4252-852b-db48d052e2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:45.517956 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-2db4b211-c549-4252-852b-db48d052e2f9 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:45.519271 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-2db4b211-c549-4252-852b-db48d052e2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:45.574877 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc060c61-947f-4fa2-93d9-a82df792205b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:45.599327 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 289/566] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:45.599327 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db4b211-c549-4252-852b-db48d052e2f9 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:45.626170 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce50bb6b-da9d-4d8f-a09b-1992326cbe35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:45.626633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50bb6b-da9d-4d8f-a09b-1992326cbe35 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:45.626984 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50bb6b-da9d-4d8f-a09b-1992326cbe35 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:45.627390 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50bb6b-da9d-4d8f-a09b-1992326cbe35 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:45.628214 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce50bb6b-da9d-4d8f-a09b-1992326cbe35 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:45.628632 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce50bb6b-da9d-4d8f-a09b-1992326cbe35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:45.630192 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce50bb6b-da9d-4d8f-a09b-1992326cbe35 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:45.647367 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 278/567] 10.222.0.41 () {56 vars in 1148 bytes} [Fri Jun 3 16:02:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:45.647367 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce50bb6b-da9d-4d8f-a09b-1992326cbe35 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:45.688581 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:45.705676 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db4b211-c549-4252-852b-db48d052e2f9 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:45.709230 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:45.744551 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d6092-8f83-4dc1-a4e7-d544378252fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:45.745595 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d6092-8f83-4dc1-a4e7-d544378252fa None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:45.746023 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d6092-8f83-4dc1-a4e7-d544378252fa None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:45.746432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d6092-8f83-4dc1-a4e7-d544378252fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:45.756199 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce50bb6b-da9d-4d8f-a09b-1992326cbe35 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:45.758021 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8d6092-8f83-4dc1-a4e7-d544378252fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:45.759741 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8d6092-8f83-4dc1-a4e7-d544378252fa None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:45.780094 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-ab8d6092-8f83-4dc1-a4e7-d544378252fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:45.780094 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:45.780094 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:45.780094 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:45.780094 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:45.780094 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:45.780094 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:45.780094 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:45.780094 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:45.780094 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:45.780094 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:45.780094 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:45.780094 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:45.780094 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:45.780094 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:45.781206 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:45.781206 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:45.781206 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:45.781206 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:45.781206 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:45.781206 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:45.781206 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:45.781206 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:45.781206 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:45.781206 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:45.781206 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:45.781206 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:45.781206 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:45.781206 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:45.781206 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:45.781206 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:45.781206 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:45.782249 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:45.782249 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:45.782249 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:45.782249 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:45.783999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 290/568] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:45.785075 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d6092-8f83-4dc1-a4e7-d544378252fa None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:45.802080 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b53ba-e418-4747-bcbc-94cdb9d64636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:45.802759 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b53ba-e418-4747-bcbc-94cdb9d64636 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:45.803043 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b53ba-e418-4747-bcbc-94cdb9d64636 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:45.803355 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b53ba-e418-4747-bcbc-94cdb9d64636 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:45.804110 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9b53ba-e418-4747-bcbc-94cdb9d64636 None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:45.804417 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9b53ba-e418-4747-bcbc-94cdb9d64636 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:45.805796 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9b53ba-e418-4747-bcbc-94cdb9d64636 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:45.814864 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 279/569] 10.222.0.41 () {56 vars in 1103 bytes} [Fri Jun 3 16:02:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:45.815799 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b53ba-e418-4747-bcbc-94cdb9d64636 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:45.897641 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d6092-8f83-4dc1-a4e7-d544378252fa None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:45.922029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b53ba-e418-4747-bcbc-94cdb9d64636 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:45.939107 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f8525-186d-4775-b164-59cbe48e83ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:45.940011 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f8525-186d-4775-b164-59cbe48e83ea None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:45.940365 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f8525-186d-4775-b164-59cbe48e83ea None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:45.940713 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f8525-186d-4775-b164-59cbe48e83ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:45.942233 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:45.950229 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:45.955284 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813f8525-186d-4775-b164-59cbe48e83ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:45.959446 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b046cd-3072-456d-9ead-fde790b0abe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:45.960104 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b046cd-3072-456d-9ead-fde790b0abe0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:45.960443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b046cd-3072-456d-9ead-fde790b0abe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:45.960724 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b046cd-3072-456d-9ead-fde790b0abe0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:45.961581 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b046cd-3072-456d-9ead-fde790b0abe0 None admin] RBAC: Adding query filter params (role.id=fe695aedbe8249a28c992a19264bc34b, scope.project.id=c274254e510a468e9acbecc6298301cc, user.id=6c5f513b781a4a73b443cded13359eda) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:45.961898 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b046cd-3072-456d-9ead-fde790b0abe0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:45.963063 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-813f8525-186d-4775-b164-59cbe48e83ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:02:45.963063 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:45.963063 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:45.963063 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:45.963063 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:45.963063 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:45.963063 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:45.963063 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:45.963063 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:45.963063 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:45.963063 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:45.963063 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:45.963063 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:45.963063 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:45.963063 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:45.963063 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:45.963975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:45.963975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:45.963975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:45.963975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:45.963975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:45.963975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:45.963975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:45.963975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:45.963975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:45.963975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:45.963975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:45.963975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:45.963975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:45.963975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:45.963975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:45.963975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:45.969988 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:45.969988 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:45.969988 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:45.969988 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:45.969988 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:45.969988 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:45.969988 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:45.969988 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:45.969988 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:45.969988 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:45.969988 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:02:45.969988 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:45.970685 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b046cd-3072-456d-9ead-fde790b0abe0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:45.970756 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 291/570] 10.222.0.41 () {56 vars in 1102 bytes} [Fri Jun 3 16:02:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:45.970756 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f8525-186d-4775-b164-59cbe48e83ea None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:45.973686 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 280/571] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:02:45 2022] GET /identity/v3/role_assignments?user.id=6c5f513b781a4a73b443cded13359eda&scope.project.id=c274254e510a468e9acbecc6298301cc&role.id=fe695aedbe8249a28c992a19264bc34b => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:45.974700 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b046cd-3072-456d-9ead-fde790b0abe0 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:46.082261 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f8525-186d-4775-b164-59cbe48e83ea None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:46.120029 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5565167a-60e0-4cd4-be7b-5fa2252f7283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:46.120710 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565167a-60e0-4cd4-be7b-5fa2252f7283 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:46.120995 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565167a-60e0-4cd4-be7b-5fa2252f7283 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:46.121271 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565167a-60e0-4cd4-be7b-5fa2252f7283 None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:46.122034 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5565167a-60e0-4cd4-be7b-5fa2252f7283 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:46.122341 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5565167a-60e0-4cd4-be7b-5fa2252f7283 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:46.123063 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b046cd-3072-456d-9ead-fde790b0abe0 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:46.123767 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5565167a-60e0-4cd4-be7b-5fa2252f7283 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:46.136562 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 292/572] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:46.139362 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5565167a-60e0-4cd4-be7b-5fa2252f7283 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:46.167947 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ced8d-0228-4cff-bd5c-dab0a16576e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:46.169546 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ced8d-0228-4cff-bd5c-dab0a16576e6 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:46.170144 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ced8d-0228-4cff-bd5c-dab0a16576e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:46.170436 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ced8d-0228-4cff-bd5c-dab0a16576e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:46.183137 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155ced8d-0228-4cff-bd5c-dab0a16576e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:46.184657 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155ced8d-0228-4cff-bd5c-dab0a16576e6 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:46.202622 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-155ced8d-0228-4cff-bd5c-dab0a16576e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:46.202622 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:46.202622 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:46.202622 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:46.202622 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:46.202622 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:46.202622 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:46.202622 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:46.202622 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:46.202622 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:46.202622 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:46.202622 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:46.202622 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:46.202622 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:46.202622 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:46.203694 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:46.203694 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:46.203694 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:46.203694 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:46.203694 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:46.203694 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:46.203694 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:46.203694 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:46.203694 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:46.203694 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:46.203694 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:46.203694 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:46.203694 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:46.203694 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:46.203694 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:46.203694 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:46.203694 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:46.204619 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:46.204619 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:46.204619 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:46.204619 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:46.204619 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 281/573] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:46.205405 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5139b22-d760-4c66-ac38-7e340851d13f None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:46.205695 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5139b22-d760-4c66-ac38-7e340851d13f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:46.205975 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5139b22-d760-4c66-ac38-7e340851d13f None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:46.208124 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 282/574] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:46.208737 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5139b22-d760-4c66-ac38-7e340851d13f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:46.214256 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:46.256252 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:46.262804 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5565167a-60e0-4cd4-be7b-5fa2252f7283 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:46.298781 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7cc9e-ef6d-4b92-ab95-1b1691ed3a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:46.299492 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7cc9e-ef6d-4b92-ab95-1b1691ed3a53 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:46.299784 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7cc9e-ef6d-4b92-ab95-1b1691ed3a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:46.300063 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7cc9e-ef6d-4b92-ab95-1b1691ed3a53 None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:46.301356 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7cc9e-ef6d-4b92-ab95-1b1691ed3a53 None admin] RBAC: Adding query filter params (name=service) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:46.301674 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7cc9e-ef6d-4b92-ab95-1b1691ed3a53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:46.303261 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7cc9e-ef6d-4b92-ab95-1b1691ed3a53 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:46.315813 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 293/575] 10.222.0.41 () {56 vars in 1103 bytes} [Fri Jun 3 16:02:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:46.315813 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7cc9e-ef6d-4b92-ab95-1b1691ed3a53 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:46.335721 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5139b22-d760-4c66-ac38-7e340851d13f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:46.372460 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f55f6a1-eeaf-4ca5-8503-261aed3cac13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:46.373759 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f55f6a1-eeaf-4ca5-8503-261aed3cac13 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:46.374207 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f55f6a1-eeaf-4ca5-8503-261aed3cac13 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:46.374486 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f55f6a1-eeaf-4ca5-8503-261aed3cac13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:46.381659 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f55f6a1-eeaf-4ca5-8503-261aed3cac13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:46.383106 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f55f6a1-eeaf-4ca5-8503-261aed3cac13 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:46.398300 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-8f55f6a1-eeaf-4ca5-8503-261aed3cac13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:02:46.398300 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:46.398300 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:46.398300 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:46.398300 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:46.398300 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:46.398300 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:46.398300 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:46.398300 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:46.398300 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:46.398300 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:46.398300 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:46.398300 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:02:46.398300 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:02:46.398300 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:02:46.399221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:02:46.399221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:46.399221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:46.399221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:46.399221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:46.399221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:46.399221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:46.399221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:46.399221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:46.399221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:46.399221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:46.399221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:46.399221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:46.399221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:46.399221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:46.399221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:46.399221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:46.400141 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:02:46.400141 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:02:46.400141 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:02:46.400141 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:02:46.400141 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:02:46.400141 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:02:46.400141 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:02:46.400141 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:46.400141 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 283/576] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:02:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:46.401160 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df385ff-5c75-4e4a-a746-aed01f3009ce None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:46.401160 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df385ff-5c75-4e4a-a746-aed01f3009ce None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:46.401405 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df385ff-5c75-4e4a-a746-aed01f3009ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:46.417631 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7cc9e-ef6d-4b92-ab95-1b1691ed3a53 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:46.419339 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-0df385ff-5c75-4e4a-a746-aed01f3009ce None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:46.420488 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0df385ff-5c75-4e4a-a746-aed01f3009ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:46.441427 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:46.457484 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e78ee-dda5-4e19-89e3-e0d1d758feac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:46.458477 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e78ee-dda5-4e19-89e3-e0d1d758feac None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:46.458897 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e78ee-dda5-4e19-89e3-e0d1d758feac None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:46.459181 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e78ee-dda5-4e19-89e3-e0d1d758feac None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:46.459954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91e78ee-dda5-4e19-89e3-e0d1d758feac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:46.460268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91e78ee-dda5-4e19-89e3-e0d1d758feac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:46.461635 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91e78ee-dda5-4e19-89e3-e0d1d758feac None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:46.480048 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 294/577] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:46.480048 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e78ee-dda5-4e19-89e3-e0d1d758feac None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:46.511327 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 284/578] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:46.511327 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df385ff-5c75-4e4a-a746-aed01f3009ce None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:46.544541 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:46.596074 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e78ee-dda5-4e19-89e3-e0d1d758feac None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:46.622589 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df385ff-5c75-4e4a-a746-aed01f3009ce None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:46.631769 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103213f-c08d-4643-92c7-131ec6496f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:46.632561 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103213f-c08d-4643-92c7-131ec6496f7f None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:46.632846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103213f-c08d-4643-92c7-131ec6496f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:46.633125 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103213f-c08d-4643-92c7-131ec6496f7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:46.644215 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5103213f-c08d-4643-92c7-131ec6496f7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:46.645647 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5103213f-c08d-4643-92c7-131ec6496f7f None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:46.657982 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c268f47a-96dd-4930-9bde-944ec3b97973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:46.658681 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c268f47a-96dd-4930-9bde-944ec3b97973 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:46.659003 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c268f47a-96dd-4930-9bde-944ec3b97973 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:46.659323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c268f47a-96dd-4930-9bde-944ec3b97973 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:46.660676 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c268f47a-96dd-4930-9bde-944ec3b97973 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:46.661015 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c268f47a-96dd-4930-9bde-944ec3b97973 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:46.662446 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c268f47a-96dd-4930-9bde-944ec3b97973 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:46.664216 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-5103213f-c08d-4643-92c7-131ec6496f7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:46.664216 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:46.664216 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:46.664216 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:46.664216 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:46.664216 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:46.664216 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:46.664216 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:46.664216 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:46.664216 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:46.664216 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:46.664216 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:46.664216 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:46.664216 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:46.664216 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:46.665078 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:46.665078 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:46.665078 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:46.665078 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:46.665078 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:46.665078 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:46.665078 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:46.665078 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:46.665078 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:46.665078 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:46.665078 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:46.665078 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:46.665078 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:46.665078 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:46.665078 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:46.665078 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:46.665078 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:46.665962 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:46.665962 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:46.665962 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:46.665962 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:46.665962 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 295/579] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:46.667021 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103213f-c08d-4643-92c7-131ec6496f7f None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:46.677480 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 285/580] 10.222.0.41 () {56 vars in 1148 bytes} [Fri Jun 3 16:02:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:46.678316 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c268f47a-96dd-4930-9bde-944ec3b97973 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:46.701297 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:46.770632 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:46.772684 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103213f-c08d-4643-92c7-131ec6496f7f None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:46.788380 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c268f47a-96dd-4930-9bde-944ec3b97973 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:46.807201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca1f5e-1044-4185-8272-7ce1a5cc5b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:46.808111 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca1f5e-1044-4185-8272-7ce1a5cc5b7b None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:46.808469 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca1f5e-1044-4185-8272-7ce1a5cc5b7b None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:46.808820 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca1f5e-1044-4185-8272-7ce1a5cc5b7b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:46.816105 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca1f5e-1044-4185-8272-7ce1a5cc5b7b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:46.817626 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca1f5e-1044-4185-8272-7ce1a5cc5b7b None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:46.824636 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a99218-f62e-4049-8d2a-adb9b3caa6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:46.825557 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a99218-f62e-4049-8d2a-adb9b3caa6e9 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:46.825916 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a99218-f62e-4049-8d2a-adb9b3caa6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:46.826264 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a99218-f62e-4049-8d2a-adb9b3caa6e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:46.833052 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-56ca1f5e-1044-4185-8272-7ce1a5cc5b7b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 03 16:02:46.833052 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:46.833052 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:46.833052 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:46.833052 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:46.833052 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:46.833052 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:46.833052 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:46.833052 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:46.833052 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:46.833052 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:46.833052 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:46.833052 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:02:46.833052 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:02:46.833052 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:02:46.833052 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:02:46.834231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:46.834231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:46.834231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:02:46.834231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:02:46.834231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:02:46.834231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:02:46.834231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:46.834231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:46.834231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:46.834231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:46.834231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:46.834231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:46.834231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:46.834231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:46.834231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:46.834231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:46.834231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:46.834231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:46.835403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:46.835403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:46.835403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:02:46.835403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:02:46.835403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:02:46.835403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:02:46.835403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:02:46.835403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:02:46.835403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 03 16:02:46.835403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:46.836239 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 296/581] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:02:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:46.837010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca1f5e-1044-4185-8272-7ce1a5cc5b7b None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:46.844306 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a99218-f62e-4049-8d2a-adb9b3caa6e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:46.852570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-d4a99218-f62e-4049-8d2a-adb9b3caa6e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:02:46.852570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:46.852570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:46.852570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:46.852570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:46.852570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:46.852570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:46.852570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:46.852570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:46.852570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:46.852570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:46.852570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:46.852570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:46.852570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:46.852570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:46.852570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:46.853496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:46.853496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:46.853496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:46.853496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:46.853496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:46.853496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:46.853496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:46.853496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:46.853496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:46.853496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:46.853496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:46.853496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:46.853496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:46.853496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:46.853496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:46.853496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:46.854334 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:46.854334 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:46.854334 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:46.854334 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:46.854334 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:46.854334 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:46.854334 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:46.854334 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:46.854334 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:46.854334 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:46.854334 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:02:46.854334 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:46.854334 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 286/582] 10.222.0.41 () {56 vars in 1102 bytes} [Fri Jun 3 16:02:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:46.855765 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a99218-f62e-4049-8d2a-adb9b3caa6e9 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:46.949212 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:46.959738 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca1f5e-1044-4185-8272-7ce1a5cc5b7b None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:46.964850 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a99218-f62e-4049-8d2a-adb9b3caa6e9 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:46.995841 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0cc1fd-01a2-4bae-937a-a113898e66b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:46.996499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0cc1fd-01a2-4bae-937a-a113898e66b4 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:46.996786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0cc1fd-01a2-4bae-937a-a113898e66b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:46.997060 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0cc1fd-01a2-4bae-937a-a113898e66b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:46.997786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0cc1fd-01a2-4bae-937a-a113898e66b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:46.998096 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0cc1fd-01a2-4bae-937a-a113898e66b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:46.999479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0cc1fd-01a2-4bae-937a-a113898e66b4 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:47.002081 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454fabb4-17e8-494a-a9b5-ef5af5f24e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:47.002960 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fabb4-17e8-494a-a9b5-ef5af5f24e02 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:47.003249 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fabb4-17e8-494a-a9b5-ef5af5f24e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:47.003582 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fabb4-17e8-494a-a9b5-ef5af5f24e02 None admin] PATH_INFO: `/v3/projects/c274254e510a468e9acbecc6298301cc/users/2fd08ea39ee94b92ac6763608e69f158/roles/fe695aedbe8249a28c992a19264bc34b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:47.014988 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 297/583] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:47.014988 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0cc1fd-01a2-4bae-937a-a113898e66b4 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:47.023350 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454fabb4-17e8-494a-a9b5-ef5af5f24e02 None admin] RBAC: Authorizing `identity:create_grant(project_id=c274254e510a468e9acbecc6298301cc, user_id=2fd08ea39ee94b92ac6763608e69f158, role_id=fe695aedbe8249a28c992a19264bc34b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:47.024879 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454fabb4-17e8-494a-a9b5-ef5af5f24e02 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:47.028554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:47.040584 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 287/584] 10.222.0.41 () {58 vars in 1439 bytes} [Fri Jun 3 16:02:46 2022] PUT /identity/v3/projects/c274254e510a468e9acbecc6298301cc/users/2fd08ea39ee94b92ac6763608e69f158/roles/fe695aedbe8249a28c992a19264bc34b => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:02:47.041600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454fabb4-17e8-494a-a9b5-ef5af5f24e02 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:47.158566 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0cc1fd-01a2-4bae-937a-a113898e66b4 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:47.182323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454fabb4-17e8-494a-a9b5-ef5af5f24e02 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:47.196695 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca81eb9b-1230-4850-a065-64daf2fdfe21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:47.197728 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81eb9b-1230-4850-a065-64daf2fdfe21 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:47.198220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81eb9b-1230-4850-a065-64daf2fdfe21 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:47.198701 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81eb9b-1230-4850-a065-64daf2fdfe21 None admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:47.199753 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca81eb9b-1230-4850-a065-64daf2fdfe21 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:47.200083 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca81eb9b-1230-4850-a065-64daf2fdfe21 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:47.201625 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca81eb9b-1230-4850-a065-64daf2fdfe21 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:47.226011 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 298/585] 10.222.0.41 () {56 vars in 1137 bytes} [Fri Jun 3 16:02:47 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:47.227196 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca81eb9b-1230-4850-a065-64daf2fdfe21 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:47.229131 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf77971-b404-47bd-a0c4-201025c2c6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:47.229968 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf77971-b404-47bd-a0c4-201025c2c6bf None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:47.230476 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf77971-b404-47bd-a0c4-201025c2c6bf None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:47.231095 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf77971-b404-47bd-a0c4-201025c2c6bf None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:47.232094 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf77971-b404-47bd-a0c4-201025c2c6bf None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:47.232554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf77971-b404-47bd-a0c4-201025c2c6bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:47.235268 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf77971-b404-47bd-a0c4-201025c2c6bf None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:47.244862 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:47.248751 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 288/586] 10.222.0.41 () {56 vars in 1103 bytes} [Fri Jun 3 16:02:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:47.251587 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf77971-b404-47bd-a0c4-201025c2c6bf None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:47.329981 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:47.371271 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca81eb9b-1230-4850-a065-64daf2fdfe21 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:47.374274 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf77971-b404-47bd-a0c4-201025c2c6bf None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:47.409319 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762e301-85be-4707-bd57-38cf5dc6ee63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:47.410652 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762e301-85be-4707-bd57-38cf5dc6ee63 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:47.410941 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762e301-85be-4707-bd57-38cf5dc6ee63 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:47.411220 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762e301-85be-4707-bd57-38cf5dc6ee63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:47.413633 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea03ec4-21f1-46c6-b769-226243ec4b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:47.414944 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea03ec4-21f1-46c6-b769-226243ec4b60 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:47.415407 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea03ec4-21f1-46c6-b769-226243ec4b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:47.415875 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea03ec4-21f1-46c6-b769-226243ec4b60 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:47.424499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea03ec4-21f1-46c6-b769-226243ec4b60 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:47.426513 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b762e301-85be-4707-bd57-38cf5dc6ee63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:47.427191 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea03ec4-21f1-46c6-b769-226243ec4b60 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:47.428128 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b762e301-85be-4707-bd57-38cf5dc6ee63 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:47.443893 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-9ea03ec4-21f1-46c6-b769-226243ec4b60 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 03 16:02:47.443893 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:47.443893 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:47.443893 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:47.443893 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:47.443893 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:47.443893 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:47.443893 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:47.443893 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:47.443893 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:47.443893 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:47.443893 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:47.443893 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:02:47.443893 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:02:47.443893 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:02:47.443893 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:02:47.445054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:47.445054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:47.445054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:02:47.445054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:02:47.445054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:02:47.445054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:02:47.445054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:47.445054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:47.445054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:47.445054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:47.445054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:47.445054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:47.445054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:47.445054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:47.445054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:47.445054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:47.445054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:47.445054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:47.446201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:47.446201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:47.446201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:02:47.446201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:02:47.446201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:02:47.446201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:02:47.446201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:02:47.446201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:02:47.446201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 03 16:02:47.446201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:47.448545 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-b762e301-85be-4707-bd57-38cf5dc6ee63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:47.448545 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:47.448545 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:47.448545 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:47.448545 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:47.448545 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:47.448545 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:47.448545 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:47.448545 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:47.448545 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:47.448545 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:47.448545 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:47.448545 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:47.448545 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:47.448545 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:47.449683 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:47.449683 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:47.449683 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:47.449683 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:47.449683 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:47.449683 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:47.449683 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:47.449683 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:47.449683 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:47.449683 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:47.449683 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:47.449683 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:47.449683 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:47.449683 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:47.449683 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:47.449683 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:47.449683 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:47.450815 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:47.450815 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:47.450815 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:47.450815 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:47.451132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 299/587] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:02:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:47.457077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea03ec4-21f1-46c6-b769-226243ec4b60 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:47.457211 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 289/588] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:47.457211 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762e301-85be-4707-bd57-38cf5dc6ee63 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:47.541891 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:47.575837 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea03ec4-21f1-46c6-b769-226243ec4b60 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:47.585801 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762e301-85be-4707-bd57-38cf5dc6ee63 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:47.607648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:47.614041 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd77b55-97d6-4aff-a1a0-5646ee3ab378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:47.614838 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd77b55-97d6-4aff-a1a0-5646ee3ab378 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:47.615126 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd77b55-97d6-4aff-a1a0-5646ee3ab378 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:47.615439 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd77b55-97d6-4aff-a1a0-5646ee3ab378 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:47.626798 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd77b55-97d6-4aff-a1a0-5646ee3ab378 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:47.628359 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd77b55-97d6-4aff-a1a0-5646ee3ab378 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:47.633858 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96676374-726b-4b10-99fb-d5d1691b9837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:47.634788 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96676374-726b-4b10-99fb-d5d1691b9837 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:47.635277 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96676374-726b-4b10-99fb-d5d1691b9837 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:47.636340 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96676374-726b-4b10-99fb-d5d1691b9837 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:47.637396 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96676374-726b-4b10-99fb-d5d1691b9837 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:47.638504 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96676374-726b-4b10-99fb-d5d1691b9837 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:47.640483 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96676374-726b-4b10-99fb-d5d1691b9837 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:47.647740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-0cd77b55-97d6-4aff-a1a0-5646ee3ab378 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:47.647740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:47.647740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:47.647740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:47.647740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:47.647740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:47.647740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:47.647740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:47.647740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:47.647740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:47.647740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:47.647740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:47.647740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:47.647740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:47.647740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:47.648634 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:47.648634 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:47.648634 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:47.648634 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:47.648634 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:47.648634 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:47.648634 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:47.648634 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:47.648634 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:47.648634 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:47.648634 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:47.648634 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:47.648634 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:47.648634 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:47.648634 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:47.648634 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:47.648634 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:47.649517 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:47.649517 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:47.649517 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:47.649517 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:47.654220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 300/589] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:47.655100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd77b55-97d6-4aff-a1a0-5646ee3ab378 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:47.668487 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 290/590] 10.222.0.41 () {56 vars in 1137 bytes} [Fri Jun 3 16:02:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:47.670020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96676374-726b-4b10-99fb-d5d1691b9837 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:47.766198 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd77b55-97d6-4aff-a1a0-5646ee3ab378 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:47.779704 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96676374-726b-4b10-99fb-d5d1691b9837 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:47.790238 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:47.804857 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8701e4-fc45-4b0f-9ef8-d9144587c926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:47.805531 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8701e4-fc45-4b0f-9ef8-d9144587c926 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:47.805820 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8701e4-fc45-4b0f-9ef8-d9144587c926 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:47.806102 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8701e4-fc45-4b0f-9ef8-d9144587c926 None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:47.807062 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8701e4-fc45-4b0f-9ef8-d9144587c926 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:47.807181 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8701e4-fc45-4b0f-9ef8-d9144587c926 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:47.808632 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8701e4-fc45-4b0f-9ef8-d9144587c926 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:47.821203 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 301/591] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:47.822182 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8701e4-fc45-4b0f-9ef8-d9144587c926 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:47.824064 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c274f599-30bc-4804-a508-2aa1cbe0d2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:47.825075 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274f599-30bc-4804-a508-2aa1cbe0d2fb None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:47.825564 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274f599-30bc-4804-a508-2aa1cbe0d2fb None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:47.826111 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274f599-30bc-4804-a508-2aa1cbe0d2fb None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:47.827058 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c274f599-30bc-4804-a508-2aa1cbe0d2fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:47.827607 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c274f599-30bc-4804-a508-2aa1cbe0d2fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:47.829197 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c274f599-30bc-4804-a508-2aa1cbe0d2fb None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:47.847649 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 291/592] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:47.848970 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c274f599-30bc-4804-a508-2aa1cbe0d2fb None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:47.879773 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:47.929872 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8701e4-fc45-4b0f-9ef8-d9144587c926 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:47.960925 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c274f599-30bc-4804-a508-2aa1cbe0d2fb None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:47.974501 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ddc0b-c2c2-4681-8ee6-01e2dc98fa9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:47.975439 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ddc0b-c2c2-4681-8ee6-01e2dc98fa9a None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:47.975735 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ddc0b-c2c2-4681-8ee6-01e2dc98fa9a None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:47.976015 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ddc0b-c2c2-4681-8ee6-01e2dc98fa9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:47.988165 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161ddc0b-c2c2-4681-8ee6-01e2dc98fa9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:47.989735 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161ddc0b-c2c2-4681-8ee6-01e2dc98fa9a None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:47.999570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f30831-230b-4ce8-aeb4-2c4376a32887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:48.000648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f30831-230b-4ce8-aeb4-2c4376a32887 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:48.001143 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f30831-230b-4ce8-aeb4-2c4376a32887 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:48.001617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f30831-230b-4ce8-aeb4-2c4376a32887 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:48.008826 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-161ddc0b-c2c2-4681-8ee6-01e2dc98fa9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:48.008826 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:48.008826 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:48.008826 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:48.008826 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:48.008826 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:48.008826 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:48.008826 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:48.008826 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:48.008826 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:48.008826 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:48.008826 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:48.008826 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:48.008826 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:48.008826 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:48.009707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:48.009707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:48.009707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:48.009707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:48.009707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:48.009707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:48.009707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:48.009707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:48.009707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:48.009707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:48.009707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:48.009707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:48.009707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:48.009707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:48.009707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:48.009707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:48.009707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:48.010606 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:48.010606 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:48.010606 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:48.010606 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:48.011755 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f30831-230b-4ce8-aeb4-2c4376a32887 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:48.013419 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f30831-230b-4ce8-aeb4-2c4376a32887 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:48.014618 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 302/593] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:48.015594 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ddc0b-c2c2-4681-8ee6-01e2dc98fa9a None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:48.030429 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-39f30831-230b-4ce8-aeb4-2c4376a32887 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:02:48.030429 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:48.030429 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:48.030429 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:48.030429 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:48.030429 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:48.030429 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:48.030429 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:48.030429 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:48.030429 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:48.030429 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:48.030429 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:48.030429 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:02:48.030429 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:02:48.030429 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:02:48.031349 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:02:48.031349 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:48.031349 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:48.031349 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:48.031349 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:48.031349 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:48.031349 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:48.031349 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:48.031349 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:48.031349 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:48.031349 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:48.031349 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:48.031349 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:48.031349 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:48.031349 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:48.031349 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:48.031349 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:48.032239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:02:48.032239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:02:48.032239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:02:48.032239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:02:48.032239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:02:48.032239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:02:48.032239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:02:48.032239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:48.034627 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 292/594] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:02:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:48.035652 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f30831-230b-4ce8-aeb4-2c4376a32887 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:48.058057 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:48.125195 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:48.134774 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ddc0b-c2c2-4681-8ee6-01e2dc98fa9a None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:48.138117 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f30831-230b-4ce8-aeb4-2c4376a32887 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:48.172560 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ad9bfe-983a-4b6e-81ce-20cbd5338ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:48.173595 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5859f5fe-a04f-4635-b0fc-38f31241aeab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:48.173915 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad9bfe-983a-4b6e-81ce-20cbd5338ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:48.174360 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad9bfe-983a-4b6e-81ce-20cbd5338ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:48.174524 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5859f5fe-a04f-4635-b0fc-38f31241aeab None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:48.174908 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5859f5fe-a04f-4635-b0fc-38f31241aeab None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:48.175110 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad9bfe-983a-4b6e-81ce-20cbd5338ae6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:48.175978 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5859f5fe-a04f-4635-b0fc-38f31241aeab None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:48.176993 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5859f5fe-a04f-4635-b0fc-38f31241aeab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:48.177399 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5859f5fe-a04f-4635-b0fc-38f31241aeab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:48.178937 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5859f5fe-a04f-4635-b0fc-38f31241aeab None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:48.184692 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ad9bfe-983a-4b6e-81ce-20cbd5338ae6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:48.186388 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ad9bfe-983a-4b6e-81ce-20cbd5338ae6 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:48.196044 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 293/595] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:48.196044 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5859f5fe-a04f-4635-b0fc-38f31241aeab None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:48.207679 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-e0ad9bfe-983a-4b6e-81ce-20cbd5338ae6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 03 16:02:48.207679 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:48.207679 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:48.207679 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:48.207679 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:48.207679 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:48.207679 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:48.207679 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:48.207679 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:48.207679 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:48.207679 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:48.207679 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:48.207679 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:02:48.207679 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:02:48.207679 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:02:48.207679 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:02:48.208723 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:48.208723 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:48.208723 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:02:48.208723 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:02:48.208723 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:02:48.208723 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:02:48.208723 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:48.208723 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:48.208723 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:48.208723 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:48.208723 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:48.208723 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:48.208723 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:48.208723 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:48.208723 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:48.208723 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:48.208723 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:48.208723 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:48.209646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:48.209646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:48.209646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:02:48.209646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:02:48.209646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:02:48.209646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:02:48.209646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:02:48.209646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:02:48.209646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 03 16:02:48.209646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:48.212308 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 303/596] 10.222.0.41 () {56 vars in 1102 bytes} [Fri Jun 3 16:02:48 2022] GET /identity/v3/users/neutron => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:48.213286 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ad9bfe-983a-4b6e-81ce-20cbd5338ae6 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:48.306692 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5859f5fe-a04f-4635-b0fc-38f31241aeab None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:48.319642 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:48.327982 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ad9bfe-983a-4b6e-81ce-20cbd5338ae6 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:48.344275 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80909d25-adae-4e49-a92f-8f78ba5813ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:48.344980 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80909d25-adae-4e49-a92f-8f78ba5813ab None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:48.345268 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80909d25-adae-4e49-a92f-8f78ba5813ab None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:48.345684 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80909d25-adae-4e49-a92f-8f78ba5813ab None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:48.346758 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80909d25-adae-4e49-a92f-8f78ba5813ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:48.347720 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80909d25-adae-4e49-a92f-8f78ba5813ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:48.349152 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80909d25-adae-4e49-a92f-8f78ba5813ab None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:48.364323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 294/597] 10.222.0.41 () {56 vars in 1148 bytes} [Fri Jun 3 16:02:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:48.365152 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80909d25-adae-4e49-a92f-8f78ba5813ab None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:48.370513 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0090f1d-86fd-4f48-841b-4eb1638d969c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:48.371435 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0090f1d-86fd-4f48-841b-4eb1638d969c None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:48.371989 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0090f1d-86fd-4f48-841b-4eb1638d969c None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:48.371989 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0090f1d-86fd-4f48-841b-4eb1638d969c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:48.377728 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:48.379327 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0090f1d-86fd-4f48-841b-4eb1638d969c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:48.380891 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0090f1d-86fd-4f48-841b-4eb1638d969c None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:48.397203 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-f0090f1d-86fd-4f48-841b-4eb1638d969c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:02:48.397203 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:48.397203 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:48.397203 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:48.397203 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:48.397203 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:48.397203 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:48.397203 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:48.397203 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:48.397203 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:48.397203 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:48.397203 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:48.397203 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:02:48.397203 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:02:48.397203 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:02:48.398087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:02:48.398087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:48.398087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:48.398087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:48.398087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:48.398087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:48.398087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:48.398087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:48.398087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:48.398087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:48.398087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:48.398087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:48.398087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:48.398087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:48.398087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:48.398087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:48.398087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:48.398942 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:02:48.398942 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:02:48.398942 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:02:48.398942 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:02:48.398942 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:02:48.398942 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:02:48.398942 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:02:48.398942 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:48.400809 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 304/598] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:02:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:48.402178 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a53b27-e468-4da7-8cef-37abae15dfc5 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:48.402591 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a53b27-e468-4da7-8cef-37abae15dfc5 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:48.403005 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a53b27-e468-4da7-8cef-37abae15dfc5 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:48.408522 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 305/599] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:48.408522 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a53b27-e468-4da7-8cef-37abae15dfc5 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:48.478661 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80909d25-adae-4e49-a92f-8f78ba5813ab None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:48.516624 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe467e-58c5-4168-91c4-1ab96ee8d79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:48.517007 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a53b27-e468-4da7-8cef-37abae15dfc5 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:48.517289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe467e-58c5-4168-91c4-1ab96ee8d79a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:48.517570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe467e-58c5-4168-91c4-1ab96ee8d79a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:48.517845 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe467e-58c5-4168-91c4-1ab96ee8d79a None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:48.519492 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efe467e-58c5-4168-91c4-1ab96ee8d79a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:48.520636 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efe467e-58c5-4168-91c4-1ab96ee8d79a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:48.522074 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efe467e-58c5-4168-91c4-1ab96ee8d79a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:48.546003 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 295/600] 10.222.0.41 () {56 vars in 1139 bytes} [Fri Jun 3 16:02:48 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:48.546003 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe467e-58c5-4168-91c4-1ab96ee8d79a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:48.557035 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9e721-d481-42bf-a281-134832b390f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:48.557861 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9e721-d481-42bf-a281-134832b390f7 None admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:48.558144 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9e721-d481-42bf-a281-134832b390f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:48.558422 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9e721-d481-42bf-a281-134832b390f7 None admin] PATH_INFO: `/v3/projects/c274254e510a468e9acbecc6298301cc/users/62a3bbb9720049b29eb37b0dd88bbad7/roles/fe695aedbe8249a28c992a19264bc34b` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:48.572716 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac9e721-d481-42bf-a281-134832b390f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=c274254e510a468e9acbecc6298301cc, user_id=62a3bbb9720049b29eb37b0dd88bbad7, role_id=fe695aedbe8249a28c992a19264bc34b)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:48.574148 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac9e721-d481-42bf-a281-134832b390f7 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:48.587070 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 306/601] 10.222.0.41 () {58 vars in 1439 bytes} [Fri Jun 3 16:02:48 2022] PUT /identity/v3/projects/c274254e510a468e9acbecc6298301cc/users/62a3bbb9720049b29eb37b0dd88bbad7/roles/fe695aedbe8249a28c992a19264bc34b => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:02:48.588562 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb613a53-c936-410b-81dd-825955da32bd None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:48.589053 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb613a53-c936-410b-81dd-825955da32bd None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:48.589544 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb613a53-c936-410b-81dd-825955da32bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:48.596685 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:48.609223 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-eb613a53-c936-410b-81dd-825955da32bd None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:48.610872 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-eb613a53-c936-410b-81dd-825955da32bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:48.630299 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:48.672873 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe467e-58c5-4168-91c4-1ab96ee8d79a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:48.717582 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 307/602] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:48.717582 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb613a53-c936-410b-81dd-825955da32bd None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:48.723999 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6860a399-3238-4569-a4d5-3a6a659873ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:48.724985 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860a399-3238-4569-a4d5-3a6a659873ec None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:48.725288 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860a399-3238-4569-a4d5-3a6a659873ec None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:48.725570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860a399-3238-4569-a4d5-3a6a659873ec None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:48.726385 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6860a399-3238-4569-a4d5-3a6a659873ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:48.726697 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6860a399-3238-4569-a4d5-3a6a659873ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:48.729448 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6860a399-3238-4569-a4d5-3a6a659873ec None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:48.752268 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 296/603] 10.222.0.41 () {56 vars in 1148 bytes} [Fri Jun 3 16:02:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:48.754053 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6860a399-3238-4569-a4d5-3a6a659873ec None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:48.841199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb613a53-c936-410b-81dd-825955da32bd None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:48.864106 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6860a399-3238-4569-a4d5-3a6a659873ec None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:48.884154 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f41f88-2a11-405c-a775-a169fdce31ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:48.885291 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f41f88-2a11-405c-a775-a169fdce31ad None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:48.886250 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f41f88-2a11-405c-a775-a169fdce31ad None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:48.886749 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f41f88-2a11-405c-a775-a169fdce31ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:48.890890 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:48.897755 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:48.901272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7249f6-4637-42ee-93eb-5805d0488641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:48.902019 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f41f88-2a11-405c-a775-a169fdce31ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:48.902803 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7249f6-4637-42ee-93eb-5805d0488641 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:48.903458 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7249f6-4637-42ee-93eb-5805d0488641 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:48.904272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7249f6-4637-42ee-93eb-5805d0488641 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:48.904600 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f41f88-2a11-405c-a775-a169fdce31ad None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:48.923109 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7249f6-4637-42ee-93eb-5805d0488641 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:48.926750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-99f41f88-2a11-405c-a775-a169fdce31ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:48.926750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:48.926750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:48.926750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:48.926750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:48.926750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:48.926750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:48.926750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:48.926750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:48.926750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:48.926750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:48.926750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:48.926750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:48.926750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:48.926750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:48.927786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:48.927786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:48.927786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:48.927786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:48.927786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:48.927786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:48.927786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:48.927786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:48.927786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:48.927786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:48.927786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:48.927786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:48.927786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:48.927786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:48.927786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:48.927786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:48.927786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:48.928803 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:48.928803 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:48.928803 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:48.928803 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:48.930880 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 308/604] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:48.931975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f41f88-2a11-405c-a775-a169fdce31ad None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:48.932894 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-6a7249f6-4637-42ee-93eb-5805d0488641 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:02:48.932894 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:48.932894 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:48.932894 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:48.932894 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:48.932894 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:48.932894 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:48.932894 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:48.932894 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:48.932894 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:48.932894 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:48.932894 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:48.932894 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:48.932894 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:48.932894 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:48.932894 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:48.933780 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:48.933780 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:48.933780 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:48.933780 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:48.933780 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:48.933780 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:48.933780 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:48.933780 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:48.933780 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:48.933780 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:48.933780 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:48.933780 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:48.933780 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:48.933780 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:48.933780 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:48.933780 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:48.934955 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:48.934955 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:48.934955 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:48.934955 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:48.934955 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:48.934955 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:48.934955 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:48.934955 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:48.934955 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:48.934955 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:48.934955 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:02:48.934955 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:48.934955 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 297/605] 10.222.0.41 () {56 vars in 1102 bytes} [Fri Jun 3 16:02:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:48.940193 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7249f6-4637-42ee-93eb-5805d0488641 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:49.044269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f41f88-2a11-405c-a775-a169fdce31ad None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:49.047274 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7249f6-4637-42ee-93eb-5805d0488641 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:49.081819 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad37588-faff-41a3-84b3-3a4e888fa5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:49.082990 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad37588-faff-41a3-84b3-3a4e888fa5c6 None admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:49.083393 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad37588-faff-41a3-84b3-3a4e888fa5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:49.083742 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad37588-faff-41a3-84b3-3a4e888fa5c6 None admin] PATH_INFO: `/v3/projects/c274254e510a468e9acbecc6298301cc/users/bbaf66d6e1f54047917cc615546bbd29/roles/fe695aedbe8249a28c992a19264bc34b` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:49.084646 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e14028-ef4a-4502-a4b7-291fecfbc958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:49.085500 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e14028-ef4a-4502-a4b7-291fecfbc958 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:49.085962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e14028-ef4a-4502-a4b7-291fecfbc958 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:49.086536 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e14028-ef4a-4502-a4b7-291fecfbc958 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:49.087570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e14028-ef4a-4502-a4b7-291fecfbc958 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:49.088072 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e14028-ef4a-4502-a4b7-291fecfbc958 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:49.089639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e14028-ef4a-4502-a4b7-291fecfbc958 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:49.101386 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad37588-faff-41a3-84b3-3a4e888fa5c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=c274254e510a468e9acbecc6298301cc, user_id=bbaf66d6e1f54047917cc615546bbd29, role_id=fe695aedbe8249a28c992a19264bc34b)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:49.103286 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad37588-faff-41a3-84b3-3a4e888fa5c6 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:49.107845 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 298/606] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:49.110582 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e14028-ef4a-4502-a4b7-291fecfbc958 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:49.120114 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 309/607] 10.222.0.41 () {58 vars in 1439 bytes} [Fri Jun 3 16:02:48 2022] PUT /identity/v3/projects/c274254e510a468e9acbecc6298301cc/users/bbaf66d6e1f54047917cc615546bbd29/roles/fe695aedbe8249a28c992a19264bc34b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:02:49.121424 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad37588-faff-41a3-84b3-3a4e888fa5c6 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:49.169632 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:49.202987 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:49.241452 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad37588-faff-41a3-84b3-3a4e888fa5c6 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:49.280918 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af3e43-db33-457c-9c3b-e49c27bf591d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:49.282194 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af3e43-db33-457c-9c3b-e49c27bf591d None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:49.282716 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af3e43-db33-457c-9c3b-e49c27bf591d None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:49.283196 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af3e43-db33-457c-9c3b-e49c27bf591d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:49.292467 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af3e43-db33-457c-9c3b-e49c27bf591d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:49.294487 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af3e43-db33-457c-9c3b-e49c27bf591d None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:49.313412 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-21af3e43-db33-457c-9c3b-e49c27bf591d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:02:49.313412 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:49.313412 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:49.313412 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:49.313412 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:49.313412 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:49.313412 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:49.313412 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:49.313412 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:49.313412 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:49.313412 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:49.313412 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:49.313412 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:02:49.313412 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:02:49.313412 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:02:49.314443 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:02:49.314443 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:49.314443 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:49.314443 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:49.314443 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:49.314443 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:49.314443 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:49.314443 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:49.314443 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:49.314443 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:49.314443 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:49.314443 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:49.314443 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:49.314443 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:49.314443 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:49.314443 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:49.314443 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:49.315338 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:02:49.315338 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:02:49.315338 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:02:49.315338 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:02:49.315338 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:02:49.315338 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:02:49.315338 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:02:49.315338 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:49.317734 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 310/608] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:02:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:49.319960 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e14028-ef4a-4502-a4b7-291fecfbc958 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:49.323485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af3e43-db33-457c-9c3b-e49c27bf591d None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:49.372860 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff59cac6-4184-4d0c-b7b4-6582a9284413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:49.373930 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59cac6-4184-4d0c-b7b4-6582a9284413 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:49.374509 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59cac6-4184-4d0c-b7b4-6582a9284413 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:49.374985 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59cac6-4184-4d0c-b7b4-6582a9284413 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:49.376006 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff59cac6-4184-4d0c-b7b4-6582a9284413 None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:49.376545 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff59cac6-4184-4d0c-b7b4-6582a9284413 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:49.379700 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff59cac6-4184-4d0c-b7b4-6582a9284413 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:49.397670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 299/609] 10.222.0.41 () {56 vars in 1103 bytes} [Fri Jun 3 16:02:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:49.404344 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff59cac6-4184-4d0c-b7b4-6582a9284413 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:49.499641 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:49.506078 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af3e43-db33-457c-9c3b-e49c27bf591d None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:49.544039 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff59cac6-4184-4d0c-b7b4-6582a9284413 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:49.553092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7574f319-0cd4-454b-a7ee-dc59c798e228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:49.554042 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574f319-0cd4-454b-a7ee-dc59c798e228 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:49.554634 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574f319-0cd4-454b-a7ee-dc59c798e228 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:49.555107 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574f319-0cd4-454b-a7ee-dc59c798e228 None admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:49.556126 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7574f319-0cd4-454b-a7ee-dc59c798e228 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:49.556623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7574f319-0cd4-454b-a7ee-dc59c798e228 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:49.557332 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:49.559400 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7574f319-0cd4-454b-a7ee-dc59c798e228 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:49.581950 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 311/610] 10.222.0.41 () {56 vars in 1148 bytes} [Fri Jun 3 16:02:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:02:49.583535 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a61cb-11d6-45f0-b595-23fdda59efea None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:49.584207 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a61cb-11d6-45f0-b595-23fdda59efea None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:49.584320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a61cb-11d6-45f0-b595-23fdda59efea None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:49.585916 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 312/611] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:49.587310 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4a61cb-11d6-45f0-b595-23fdda59efea None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:49.589884 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f587db69-e71e-4858-bc91-420daae0a2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:49.591117 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587db69-e71e-4858-bc91-420daae0a2b2 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:49.591582 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587db69-e71e-4858-bc91-420daae0a2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:49.592048 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587db69-e71e-4858-bc91-420daae0a2b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:49.605086 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f587db69-e71e-4858-bc91-420daae0a2b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:49.606751 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f587db69-e71e-4858-bc91-420daae0a2b2 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:49.625078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-f587db69-e71e-4858-bc91-420daae0a2b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:49.625078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:49.625078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:49.625078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:49.625078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:49.625078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:49.625078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:49.625078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:49.625078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:49.625078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:49.625078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:49.625078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:49.625078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:49.625078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:49.625078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:49.626147 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:49.626147 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:49.626147 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:49.626147 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:49.626147 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:49.626147 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:49.626147 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:49.626147 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:49.626147 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:49.626147 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:49.626147 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:49.626147 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:49.626147 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:49.626147 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:49.626147 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:49.626147 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:49.626147 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:49.627058 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:49.627058 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:49.627058 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:49.627058 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:49.628842 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 300/612] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:49.630345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9b8c1-ebcf-49de-b09b-486b1843740a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:49.630818 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9b8c1-ebcf-49de-b09b-486b1843740a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:49.631276 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9b8c1-ebcf-49de-b09b-486b1843740a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:49.648898 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-d4b9b8c1-ebcf-49de-b09b-486b1843740a None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:49.650338 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d4b9b8c1-ebcf-49de-b09b-486b1843740a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:49.712632 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4a61cb-11d6-45f0-b595-23fdda59efea None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:49.731712 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 301/613] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:49.732707 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9b8c1-ebcf-49de-b09b-486b1843740a None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:49.761538 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97096932-9ad5-4d26-87a0-12794211b81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:49.763017 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97096932-9ad5-4d26-87a0-12794211b81a None admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:49.763927 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97096932-9ad5-4d26-87a0-12794211b81a None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:49.764673 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97096932-9ad5-4d26-87a0-12794211b81a None admin] PATH_INFO: `/v3/projects/c274254e510a468e9acbecc6298301cc/users/aeeaad0680074179b3d29abcf0d51515/roles/fe695aedbe8249a28c992a19264bc34b` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:49.782041 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97096932-9ad5-4d26-87a0-12794211b81a None admin] RBAC: Authorizing `identity:create_grant(project_id=c274254e510a468e9acbecc6298301cc, user_id=aeeaad0680074179b3d29abcf0d51515, role_id=fe695aedbe8249a28c992a19264bc34b)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:49.783535 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97096932-9ad5-4d26-87a0-12794211b81a None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:49.797824 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 313/614] 10.222.0.41 () {58 vars in 1439 bytes} [Fri Jun 3 16:02:49 2022] PUT /identity/v3/projects/c274254e510a468e9acbecc6298301cc/users/aeeaad0680074179b3d29abcf0d51515/roles/fe695aedbe8249a28c992a19264bc34b => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:02:49.798711 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97096932-9ad5-4d26-87a0-12794211b81a None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:49.808824 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:49.841476 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:49.866708 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9b8c1-ebcf-49de-b09b-486b1843740a None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:49.936926 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e21f6b-f6ca-4863-86a2-ae0ae1956bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:49.937362 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97096932-9ad5-4d26-87a0-12794211b81a None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:49.937611 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e21f6b-f6ca-4863-86a2-ae0ae1956bbd None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:49.937901 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e21f6b-f6ca-4863-86a2-ae0ae1956bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:49.938179 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e21f6b-f6ca-4863-86a2-ae0ae1956bbd None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:49.938946 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e21f6b-f6ca-4863-86a2-ae0ae1956bbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:49.939235 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e21f6b-f6ca-4863-86a2-ae0ae1956bbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:49.940910 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e21f6b-f6ca-4863-86a2-ae0ae1956bbd None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:49.965886 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 302/615] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:49.970916 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e21f6b-f6ca-4863-86a2-ae0ae1956bbd None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:49.982846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33387899-d4ed-4967-bc4e-445b590332a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:49.984089 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33387899-d4ed-4967-bc4e-445b590332a6 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:49.985235 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33387899-d4ed-4967-bc4e-445b590332a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:49.985730 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33387899-d4ed-4967-bc4e-445b590332a6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:49.999717 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33387899-d4ed-4967-bc4e-445b590332a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:50.009853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-33387899-d4ed-4967-bc4e-445b590332a6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:02:50.009853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:50.009853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:50.009853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:50.009853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:50.009853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:50.009853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:50.009853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:50.009853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:50.009853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:50.009853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:50.009853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:50.009853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:50.009853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:50.009853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:50.009853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:50.010862 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:50.010862 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:50.010862 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:50.010862 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:50.010862 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:50.010862 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:50.010862 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:50.010862 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:50.010862 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:50.010862 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:50.010862 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:50.010862 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:50.010862 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:50.010862 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:50.010862 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:50.010862 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:50.011742 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:50.011742 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:50.011742 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:50.011742 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:50.011742 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:50.011742 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:50.011742 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:50.011742 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:50.011742 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:50.011742 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:50.011742 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:02:50.011742 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:50.015082 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 314/616] 10.222.0.41 () {56 vars in 1102 bytes} [Fri Jun 3 16:02:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:50.024096 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33387899-d4ed-4967-bc4e-445b590332a6 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:50.113682 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e21f6b-f6ca-4863-86a2-ae0ae1956bbd None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:50.136251 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:50.149366 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc5afb9-21db-435b-aa74-fe162c092251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:50.150161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5afb9-21db-435b-aa74-fe162c092251 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:50.150450 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5afb9-21db-435b-aa74-fe162c092251 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:50.151000 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5afb9-21db-435b-aa74-fe162c092251 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:50.156534 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33387899-d4ed-4967-bc4e-445b590332a6 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:50.159273 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc5afb9-21db-435b-aa74-fe162c092251 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:50.161044 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc5afb9-21db-435b-aa74-fe162c092251 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:50.175550 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-fcc5afb9-21db-435b-aa74-fe162c092251 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 03 16:02:50.175550 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:50.175550 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:50.175550 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:50.175550 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:50.175550 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:50.175550 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:50.175550 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:50.175550 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:50.175550 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:50.175550 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:50.175550 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:50.175550 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:02:50.175550 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:02:50.175550 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:02:50.175550 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:02:50.176515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:50.176515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:50.176515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:02:50.176515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:02:50.176515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:02:50.176515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:02:50.176515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:50.176515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:50.176515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:50.176515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:50.176515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:50.176515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:50.176515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:50.176515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:50.176515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:50.176515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:50.176515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:50.176515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:50.177452 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:50.177452 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:50.177452 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:02:50.177452 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:02:50.177452 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:02:50.177452 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:02:50.177452 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:02:50.177452 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:02:50.177452 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 03 16:02:50.177452 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:50.177452 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 303/617] 10.222.0.41 () {56 vars in 1096 bytes} [Fri Jun 3 16:02:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:02:50.182502 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc5afb9-21db-435b-aa74-fe162c092251 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:50.195039 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb63808-7f6d-4d8a-adbd-2dee9c6b83b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:50.195916 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb63808-7f6d-4d8a-adbd-2dee9c6b83b7 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:50.196214 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb63808-7f6d-4d8a-adbd-2dee9c6b83b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:50.196505 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb63808-7f6d-4d8a-adbd-2dee9c6b83b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:50.197270 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb63808-7f6d-4d8a-adbd-2dee9c6b83b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:50.197616 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb63808-7f6d-4d8a-adbd-2dee9c6b83b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:50.199561 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:50.200527 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb63808-7f6d-4d8a-adbd-2dee9c6b83b7 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:50.211189 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 315/618] 10.222.0.41 () {56 vars in 1103 bytes} [Fri Jun 3 16:02:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:02:50.216801 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb63808-7f6d-4d8a-adbd-2dee9c6b83b7 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:50.315197 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc5afb9-21db-435b-aa74-fe162c092251 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:50.361428 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e8166-948e-4124-bd89-547e962a236f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:50.362114 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e8166-948e-4124-bd89-547e962a236f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:50.362402 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e8166-948e-4124-bd89-547e962a236f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:50.362720 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e8166-948e-4124-bd89-547e962a236f None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:50.363598 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e8166-948e-4124-bd89-547e962a236f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:50.363919 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e8166-948e-4124-bd89-547e962a236f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:50.365362 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e8166-948e-4124-bd89-547e962a236f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:50.397960 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 304/619] 10.222.0.41 () {56 vars in 1135 bytes} [Fri Jun 3 16:02:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:50.402960 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e8166-948e-4124-bd89-547e962a236f None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:50.440715 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb63808-7f6d-4d8a-adbd-2dee9c6b83b7 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:50.479750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:50.487972 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e16afdc-36fc-4438-860b-519a769e07da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:50.488921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16afdc-36fc-4438-860b-519a769e07da None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:50.489305 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16afdc-36fc-4438-860b-519a769e07da None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:50.489598 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16afdc-36fc-4438-860b-519a769e07da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:50.510239 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e16afdc-36fc-4438-860b-519a769e07da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:50.511745 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e16afdc-36fc-4438-860b-519a769e07da None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:50.526686 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:50.540504 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-8e16afdc-36fc-4438-860b-519a769e07da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:50.540504 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:50.540504 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:50.540504 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:50.540504 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:50.540504 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:50.540504 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:50.540504 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:50.540504 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:50.540504 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:50.540504 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:50.540504 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:50.540504 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:50.540504 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:50.540504 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:50.541471 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:50.541471 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:50.541471 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:50.541471 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:50.541471 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:50.541471 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:50.541471 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:50.541471 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:50.541471 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:50.541471 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:50.541471 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:50.541471 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:50.541471 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:50.541471 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:50.541471 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:50.541471 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:50.541471 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:50.542347 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:50.542347 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:50.542347 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:50.542347 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:50.544997 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 316/620] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:50.551837 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e16afdc-36fc-4438-860b-519a769e07da None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:50.593537 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e8166-948e-4124-bd89-547e962a236f None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:50.655497 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81e208-20b9-48a5-bc4f-11eded13cfbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:50.656295 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81e208-20b9-48a5-bc4f-11eded13cfbb None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:50.656579 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81e208-20b9-48a5-bc4f-11eded13cfbb None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:50.656851 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81e208-20b9-48a5-bc4f-11eded13cfbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:50.678810 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc81e208-20b9-48a5-bc4f-11eded13cfbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:50.680315 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc81e208-20b9-48a5-bc4f-11eded13cfbb None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:50.720185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-cc81e208-20b9-48a5-bc4f-11eded13cfbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:50.720185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:50.720185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:50.720185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:50.720185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:50.720185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:50.720185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:50.720185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:50.720185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:50.720185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:50.720185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:50.720185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:50.720185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:50.720185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:50.720185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:50.721119 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:50.721119 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:50.721119 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:50.721119 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:50.721119 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:50.721119 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:50.721119 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:50.721119 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:50.721119 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:50.721119 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:50.721119 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:50.721119 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:50.721119 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:50.721119 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:50.721119 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:50.721119 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:50.721119 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:50.722047 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:50.722047 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:50.722047 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:50.722047 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:50.722857 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 305/621] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:02:50.726887 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81e208-20b9-48a5-bc4f-11eded13cfbb None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:50.749806 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e16afdc-36fc-4438-860b-519a769e07da None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:50.817839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec4c13-533a-483b-8503-be7eccf5ac75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:50.818519 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec4c13-533a-483b-8503-be7eccf5ac75 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:50.818801 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec4c13-533a-483b-8503-be7eccf5ac75 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:50.819080 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec4c13-533a-483b-8503-be7eccf5ac75 None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:50.819840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ec4c13-533a-483b-8503-be7eccf5ac75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:50.820155 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ec4c13-533a-483b-8503-be7eccf5ac75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:50.822511 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ec4c13-533a-483b-8503-be7eccf5ac75 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:50.847921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 317/622] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:50.856248 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec4c13-533a-483b-8503-be7eccf5ac75 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:50.925719 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81e208-20b9-48a5-bc4f-11eded13cfbb None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:50.929038 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:50.969317 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:51.002504 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16981001-c7a3-430d-bfda-f083025633b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:51.003090 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16981001-c7a3-430d-bfda-f083025633b8 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:51.003461 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16981001-c7a3-430d-bfda-f083025633b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:51.003742 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16981001-c7a3-430d-bfda-f083025633b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:51.004471 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16981001-c7a3-430d-bfda-f083025633b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:51.004782 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16981001-c7a3-430d-bfda-f083025633b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:51.006154 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16981001-c7a3-430d-bfda-f083025633b8 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:51.029787 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 306/623] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:51.036126 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16981001-c7a3-430d-bfda-f083025633b8 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:51.054124 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec4c13-533a-483b-8503-be7eccf5ac75 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:51.114611 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ccd4c-09b7-47fe-8dec-0a450d4102b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:51.115454 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ccd4c-09b7-47fe-8dec-0a450d4102b3 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:51.115744 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ccd4c-09b7-47fe-8dec-0a450d4102b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:51.116017 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ccd4c-09b7-47fe-8dec-0a450d4102b3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:51.131934 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ccd4c-09b7-47fe-8dec-0a450d4102b3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:51.133418 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ccd4c-09b7-47fe-8dec-0a450d4102b3 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:51.148927 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-8f5ccd4c-09b7-47fe-8dec-0a450d4102b3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 03 16:02:51.148927 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:51.148927 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:51.148927 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:51.148927 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:51.148927 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:51.148927 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:51.148927 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:51.148927 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:51.148927 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:51.148927 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:51.148927 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:51.148927 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:02:51.148927 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:02:51.148927 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:02:51.148927 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:02:51.149899 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:51.149899 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:51.149899 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:02:51.149899 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:02:51.149899 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:02:51.149899 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:02:51.149899 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:51.149899 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:51.149899 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:51.149899 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:51.149899 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:51.149899 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:51.149899 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:51.149899 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:51.149899 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:51.149899 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:51.149899 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:51.149899 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:51.151477 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:51.151477 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:51.151477 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:02:51.151477 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:02:51.151477 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:02:51.151477 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:02:51.151477 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:02:51.151477 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:02:51.151477 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 03 16:02:51.151477 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:51.151477 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 318/624] 10.222.0.41 () {56 vars in 1093 bytes} [Fri Jun 3 16:02:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:51.152072 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834511a7-d629-4a16-972b-b1455f80c8df None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:51.152072 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834511a7-d629-4a16-972b-b1455f80c8df None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:51.152422 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834511a7-d629-4a16-972b-b1455f80c8df None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:51.157955 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 319/625] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:51.159179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8fd497-4391-42f6-a5de-960384c3c4f6 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:51.160016 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8fd497-4391-42f6-a5de-960384c3c4f6 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:51.160297 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8fd497-4391-42f6-a5de-960384c3c4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:51.177306 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-0c8fd497-4391-42f6-a5de-960384c3c4f6 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:51.178488 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-0c8fd497-4391-42f6-a5de-960384c3c4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:51.190264 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16981001-c7a3-430d-bfda-f083025633b8 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:51.238223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff780122-d0cc-4f5b-ade7-224f768c5d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:51.239588 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff780122-d0cc-4f5b-ade7-224f768c5d08 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:51.239879 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff780122-d0cc-4f5b-ade7-224f768c5d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:51.240162 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff780122-d0cc-4f5b-ade7-224f768c5d08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:51.253203 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff780122-d0cc-4f5b-ade7-224f768c5d08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:51.254845 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff780122-d0cc-4f5b-ade7-224f768c5d08 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:51.256930 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 320/626] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:51.264199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8fd497-4391-42f6-a5de-960384c3c4f6 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:51.272265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-ff780122-d0cc-4f5b-ade7-224f768c5d08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:02:51.272265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:51.272265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:51.272265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:51.272265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:51.272265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:51.272265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:51.272265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:51.272265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:51.272265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:51.272265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:51.272265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:51.272265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:02:51.272265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:02:51.272265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:02:51.273358 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:02:51.273358 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:51.273358 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:51.273358 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:51.273358 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:51.273358 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:51.273358 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:51.273358 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:51.273358 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:51.273358 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:51.273358 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:51.273358 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:51.273358 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:51.273358 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:51.273358 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:51.273358 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:51.273358 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:51.274453 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:02:51.274453 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:02:51.274453 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:02:51.274453 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:02:51.274453 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:02:51.274453 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:02:51.274453 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:02:51.274453 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:51.276727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 307/627] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:02:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:51.282918 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff780122-d0cc-4f5b-ade7-224f768c5d08 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:51.335907 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:51.355740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:51.391699 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8fd497-4391-42f6-a5de-960384c3c4f6 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:51.413606 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff780122-d0cc-4f5b-ade7-224f768c5d08 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:51.436562 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989b376-9abd-4695-8daa-4594fe6ff2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:51.437716 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989b376-9abd-4695-8daa-4594fe6ff2e4 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:51.438527 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989b376-9abd-4695-8daa-4594fe6ff2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:51.439158 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989b376-9abd-4695-8daa-4594fe6ff2e4 None admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:51.441199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989b376-9abd-4695-8daa-4594fe6ff2e4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:51.441517 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989b376-9abd-4695-8daa-4594fe6ff2e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:51.442945 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989b376-9abd-4695-8daa-4594fe6ff2e4 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:51.454326 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c902384-f82f-4112-bdb1-553c1d9eb0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:51.455259 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c902384-f82f-4112-bdb1-553c1d9eb0e7 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:51.455775 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c902384-f82f-4112-bdb1-553c1d9eb0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:51.456804 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c902384-f82f-4112-bdb1-553c1d9eb0e7 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:51.457796 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c902384-f82f-4112-bdb1-553c1d9eb0e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:51.458892 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c902384-f82f-4112-bdb1-553c1d9eb0e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:51.460843 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c902384-f82f-4112-bdb1-553c1d9eb0e7 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:51.462128 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 321/628] 10.222.0.41 () {56 vars in 1133 bytes} [Fri Jun 3 16:02:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:51.463060 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989b376-9abd-4695-8daa-4594fe6ff2e4 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:51.483411 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 308/629] 10.222.0.41 () {56 vars in 1148 bytes} [Fri Jun 3 16:02:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:51.484700 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c902384-f82f-4112-bdb1-553c1d9eb0e7 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:51.560590 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:51.573619 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989b376-9abd-4695-8daa-4594fe6ff2e4 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:51.593613 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:51.598487 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c902384-f82f-4112-bdb1-553c1d9eb0e7 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:51.611657 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7727a325-cd46-433f-a63c-b615d93fcdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:51.612461 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727a325-cd46-433f-a63c-b615d93fcdb7 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:51.613369 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727a325-cd46-433f-a63c-b615d93fcdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:51.613645 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727a325-cd46-433f-a63c-b615d93fcdb7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:51.624882 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7727a325-cd46-433f-a63c-b615d93fcdb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:51.633389 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-7727a325-cd46-433f-a63c-b615d93fcdb7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:02:51.633389 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:51.633389 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:51.633389 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:51.633389 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:51.633389 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:51.633389 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:51.633389 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:51.633389 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:51.633389 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:51.633389 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:51.633389 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:51.633389 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:51.633389 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:51.633389 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:51.633389 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:51.634416 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:51.634416 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:51.634416 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:51.634416 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:51.634416 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:51.634416 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:51.634416 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:51.634416 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:51.634416 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:51.634416 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:51.634416 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:51.634416 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:51.634416 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:51.634416 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:51.634416 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:51.634416 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:51.635235 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:51.635235 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:51.635235 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:51.635235 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:51.635235 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:51.635235 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:51.635235 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:51.635235 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:51.635235 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:51.635235 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:51.635235 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:02:51.635235 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:51.639403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 322/630] 10.222.0.41 () {56 vars in 1102 bytes} [Fri Jun 3 16:02:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:51.640866 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7727a325-cd46-433f-a63c-b615d93fcdb7 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:51.644730 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4b63c-cb82-47f5-8900-8941b793018c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:51.645412 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4b63c-cb82-47f5-8900-8941b793018c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:51.645698 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4b63c-cb82-47f5-8900-8941b793018c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:51.645982 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4b63c-cb82-47f5-8900-8941b793018c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:51.720853 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d4b63c-cb82-47f5-8900-8941b793018c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:51.723050 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d4b63c-cb82-47f5-8900-8941b793018c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:51.756819 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7727a325-cd46-433f-a63c-b615d93fcdb7 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:51.795799 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec1800-3d43-4dc6-b9a7-aae6e98103f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:51.797071 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec1800-3d43-4dc6-b9a7-aae6e98103f4 None admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:51.797579 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec1800-3d43-4dc6-b9a7-aae6e98103f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:51.798053 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec1800-3d43-4dc6-b9a7-aae6e98103f4 None admin] PATH_INFO: `/v3/projects/c274254e510a468e9acbecc6298301cc/users/6c5f513b781a4a73b443cded13359eda/roles/fe695aedbe8249a28c992a19264bc34b` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:51.814984 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec1800-3d43-4dc6-b9a7-aae6e98103f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=c274254e510a468e9acbecc6298301cc, user_id=6c5f513b781a4a73b443cded13359eda, role_id=fe695aedbe8249a28c992a19264bc34b)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:51.816831 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec1800-3d43-4dc6-b9a7-aae6e98103f4 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:51.826203 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:51.828365 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:51.838125 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 323/631] 10.222.0.41 () {58 vars in 1439 bytes} [Fri Jun 3 16:02:51 2022] PUT /identity/v3/projects/c274254e510a468e9acbecc6298301cc/users/6c5f513b781a4a73b443cded13359eda/roles/fe695aedbe8249a28c992a19264bc34b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:02:51.843647 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 309/632] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:02:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 16:02:51.843997 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec1800-3d43-4dc6-b9a7-aae6e98103f4 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:51.844466 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4b63c-cb82-47f5-8900-8941b793018c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:52.054974 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4b63c-cb82-47f5-8900-8941b793018c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:52.107252 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6db7e5-95c9-48ff-ae4a-aa1162721f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:52.108664 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6db7e5-95c9-48ff-ae4a-aa1162721f32 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:52.109604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6db7e5-95c9-48ff-ae4a-aa1162721f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:52.110333 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6db7e5-95c9-48ff-ae4a-aa1162721f32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:52.117034 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec1800-3d43-4dc6-b9a7-aae6e98103f4 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:52.123412 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6db7e5-95c9-48ff-ae4a-aa1162721f32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:52.125137 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6db7e5-95c9-48ff-ae4a-aa1162721f32 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:52.147501 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-7f6db7e5-95c9-48ff-ae4a-aa1162721f32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:52.147501 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:52.147501 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:52.147501 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:52.147501 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:52.147501 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:52.147501 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:52.147501 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:52.147501 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:52.147501 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:52.147501 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:52.147501 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:52.147501 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:52.147501 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:52.147501 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:52.148427 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:52.148427 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:52.148427 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:52.148427 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:52.148427 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:52.148427 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:52.148427 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:52.148427 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:52.148427 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:52.148427 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:52.148427 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:52.148427 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:52.148427 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:52.148427 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:52.148427 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:52.148427 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:52.148427 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:52.149288 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:52.149288 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:52.149288 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:52.149288 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:52.151780 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 310/633] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:52.158667 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf46ce4-b951-426b-a1c4-636d7fba2018 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:52.159264 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf46ce4-b951-426b-a1c4-636d7fba2018 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:52.160383 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf46ce4-b951-426b-a1c4-636d7fba2018 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:52.163064 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 311/634] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:52 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:52.164060 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf46ce4-b951-426b-a1c4-636d7fba2018 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:52.174567 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b57d9d6-e3eb-40b7-87c1-a403ba88f7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:52.175418 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57d9d6-e3eb-40b7-87c1-a403ba88f7ee None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:52.176100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57d9d6-e3eb-40b7-87c1-a403ba88f7ee None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:52.176100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57d9d6-e3eb-40b7-87c1-a403ba88f7ee None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:52.176918 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b57d9d6-e3eb-40b7-87c1-a403ba88f7ee None admin] RBAC: Adding query filter params (name=service) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:52.177316 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b57d9d6-e3eb-40b7-87c1-a403ba88f7ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:52.180585 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b57d9d6-e3eb-40b7-87c1-a403ba88f7ee None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:52.191244 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 324/635] 10.222.0.41 () {56 vars in 1103 bytes} [Fri Jun 3 16:02:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:52.195469 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619dce20-8c17-4bc4-a4d3-7224ef9faac1 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:52.195469 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619dce20-8c17-4bc4-a4d3-7224ef9faac1 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:52.195469 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619dce20-8c17-4bc4-a4d3-7224ef9faac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:52.214902 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-619dce20-8c17-4bc4-a4d3-7224ef9faac1 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:52.217753 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-619dce20-8c17-4bc4-a4d3-7224ef9faac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:52.242490 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:52.316835 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:52.327181 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf46ce4-b951-426b-a1c4-636d7fba2018 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:52.338818 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 325/636] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:52.339777 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619dce20-8c17-4bc4-a4d3-7224ef9faac1 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:52.380710 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94557d46-0d92-4800-b955-ed37850aec27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:52.382403 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94557d46-0d92-4800-b955-ed37850aec27 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:52.382704 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94557d46-0d92-4800-b955-ed37850aec27 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:52.382991 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94557d46-0d92-4800-b955-ed37850aec27 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:52.383775 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94557d46-0d92-4800-b955-ed37850aec27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:52.384089 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94557d46-0d92-4800-b955-ed37850aec27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:52.385478 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94557d46-0d92-4800-b955-ed37850aec27 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:52.402714 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 312/637] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:52.404309 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38d053-a0c8-4b8b-84b4-8df5fc0009c3 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:52.404601 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38d053-a0c8-4b8b-84b4-8df5fc0009c3 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:52.404879 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38d053-a0c8-4b8b-84b4-8df5fc0009c3 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:52.406265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 313/638] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:52.406793 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38d053-a0c8-4b8b-84b4-8df5fc0009c3 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:52.477412 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619dce20-8c17-4bc4-a4d3-7224ef9faac1 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:52.512678 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82140b-99eb-4295-9199-5ec4aca09ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:52.513482 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82140b-99eb-4295-9199-5ec4aca09ad4 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:52.513773 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82140b-99eb-4295-9199-5ec4aca09ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:52.514062 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82140b-99eb-4295-9199-5ec4aca09ad4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:52.525017 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f82140b-99eb-4295-9199-5ec4aca09ad4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:52.526482 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f82140b-99eb-4295-9199-5ec4aca09ad4 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:52.533118 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38d053-a0c8-4b8b-84b4-8df5fc0009c3 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:52.546268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-1f82140b-99eb-4295-9199-5ec4aca09ad4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:52.546268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:52.546268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:52.546268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:52.546268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:52.546268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:52.546268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:52.546268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:52.546268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:52.546268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:52.546268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:52.546268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:52.546268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:52.546268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:52.546268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:52.547278 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:52.547278 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:52.547278 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:52.547278 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:52.547278 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:52.547278 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:52.547278 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:52.547278 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:52.547278 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:52.547278 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:52.547278 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:52.547278 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:52.547278 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:52.547278 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:52.547278 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:52.547278 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:52.547278 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:52.553687 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:52.553687 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:52.553687 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:52.553687 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:52.553975 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:52.554065 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 326/639] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:52.554065 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77995b8d-bd20-4577-ad84-fa7b9a27768c None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:52.554065 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77995b8d-bd20-4577-ad84-fa7b9a27768c None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:52.554065 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77995b8d-bd20-4577-ad84-fa7b9a27768c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:52.568745 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-77995b8d-bd20-4577-ad84-fa7b9a27768c None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:52.569928 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-77995b8d-bd20-4577-ad84-fa7b9a27768c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:52.576013 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a34784a-eabf-4638-83ec-550ebee8a01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:52.576818 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34784a-eabf-4638-83ec-550ebee8a01d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:52.577104 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34784a-eabf-4638-83ec-550ebee8a01d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:52.577384 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34784a-eabf-4638-83ec-550ebee8a01d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:52.589125 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a34784a-eabf-4638-83ec-550ebee8a01d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:52.597570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-5a34784a-eabf-4638-83ec-550ebee8a01d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:02:52.597570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:52.597570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:52.597570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:52.597570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:52.597570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:52.597570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:52.597570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:52.597570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:52.597570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:52.597570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:52.597570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:52.597570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:52.597570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:52.597570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:52.597570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:52.598478 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:52.598478 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:52.598478 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:52.598478 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:52.598478 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:52.598478 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:52.598478 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:52.598478 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:52.598478 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:52.598478 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:52.598478 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:52.598478 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:52.598478 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:52.598478 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:52.598478 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:52.598478 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:52.599360 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:52.599360 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:52.599360 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:52.599360 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:52.599360 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:52.599360 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:52.599360 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:52.599360 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:52.599360 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:52.599360 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:52.599360 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:02:52.599360 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:52.605881 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 314/640] 10.222.0.41 () {56 vars in 1102 bytes} [Fri Jun 3 16:02:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:52.605881 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a34784a-eabf-4638-83ec-550ebee8a01d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:52.613407 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:52.650142 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 327/641] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:52.655588 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77995b8d-bd20-4577-ad84-fa7b9a27768c None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:52.716529 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a34784a-eabf-4638-83ec-550ebee8a01d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:52.750538 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45f6c7-6d29-4aca-9e67-c11d7f408cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:52.752054 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45f6c7-6d29-4aca-9e67-c11d7f408cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:52.752484 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45f6c7-6d29-4aca-9e67-c11d7f408cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:52.752768 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45f6c7-6d29-4aca-9e67-c11d7f408cf0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:52.759815 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba45f6c7-6d29-4aca-9e67-c11d7f408cf0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:52.761258 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba45f6c7-6d29-4aca-9e67-c11d7f408cf0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:52.763216 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77995b8d-bd20-4577-ad84-fa7b9a27768c None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:52.776193 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-ba45f6c7-6d29-4aca-9e67-c11d7f408cf0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:02:52.776193 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:52.776193 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:52.776193 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:52.776193 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:52.776193 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:52.776193 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:52.776193 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:52.776193 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:52.776193 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:52.776193 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:52.776193 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:52.776193 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:02:52.776193 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:02:52.776193 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:02:52.777161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:02:52.777161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:52.777161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:52.777161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:52.777161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:52.777161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:52.777161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:52.777161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:52.777161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:52.777161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:52.777161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:52.777161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:52.777161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:52.777161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:52.777161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:52.777161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:52.777161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:52.778040 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:02:52.778040 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:02:52.778040 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:02:52.778040 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:02:52.778040 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:02:52.778040 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:02:52.778040 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:02:52.778040 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:52.782473 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 315/642] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:02:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:52.783423 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45f6c7-6d29-4aca-9e67-c11d7f408cf0 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:52.800749 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a981f5f-97e8-41ef-a19a-321ac2a51ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:52.801414 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a981f5f-97e8-41ef-a19a-321ac2a51ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:52.801700 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a981f5f-97e8-41ef-a19a-321ac2a51ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:52.801976 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a981f5f-97e8-41ef-a19a-321ac2a51ea1 None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:52.802695 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a981f5f-97e8-41ef-a19a-321ac2a51ea1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:52.803007 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a981f5f-97e8-41ef-a19a-321ac2a51ea1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:52.804438 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a981f5f-97e8-41ef-a19a-321ac2a51ea1 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:52.816429 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:52.816572 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 328/643] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:52.817427 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a981f5f-97e8-41ef-a19a-321ac2a51ea1 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:52.850397 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:52.892148 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45f6c7-6d29-4aca-9e67-c11d7f408cf0 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:52.923943 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a981f5f-97e8-41ef-a19a-321ac2a51ea1 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:52.929874 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af6e03-0f30-4402-a69b-12fc3e7cf9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:52.930537 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af6e03-0f30-4402-a69b-12fc3e7cf9b9 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:52.930821 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af6e03-0f30-4402-a69b-12fc3e7cf9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:52.931608 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af6e03-0f30-4402-a69b-12fc3e7cf9b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:52.932371 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9af6e03-0f30-4402-a69b-12fc3e7cf9b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:52.932684 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9af6e03-0f30-4402-a69b-12fc3e7cf9b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:52.934907 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9af6e03-0f30-4402-a69b-12fc3e7cf9b9 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:52.945275 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 316/644] 10.222.0.41 () {56 vars in 1103 bytes} [Fri Jun 3 16:02:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:52.946140 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af6e03-0f30-4402-a69b-12fc3e7cf9b9 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:52.965542 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6a321e-5734-47d2-8624-76b1d3afb4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:52.967166 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a321e-5734-47d2-8624-76b1d3afb4f0 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:52.967679 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a321e-5734-47d2-8624-76b1d3afb4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:52.968155 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a321e-5734-47d2-8624-76b1d3afb4f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:52.983163 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6a321e-5734-47d2-8624-76b1d3afb4f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:52.991724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-1c6a321e-5734-47d2-8624-76b1d3afb4f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:02:52.991724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:52.991724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:52.991724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:52.991724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:52.991724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:52.991724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:52.991724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:52.991724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:52.991724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:52.991724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:52.991724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:52.991724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:52.991724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:52.991724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:52.991724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:52.992668 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:52.992668 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:52.992668 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:52.992668 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:52.992668 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:52.992668 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:52.992668 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:52.992668 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:52.992668 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:52.992668 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:52.992668 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:52.992668 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:52.992668 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:52.992668 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:52.992668 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:52.992668 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:52.993628 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:52.993628 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:52.993628 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:52.993628 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:52.993628 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:52.993628 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:52.993628 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:52.993628 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:52.993628 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:52.993628 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:52.993628 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:02:52.993628 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:52.993628 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 329/645] 10.222.0.41 () {56 vars in 1102 bytes} [Fri Jun 3 16:02:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:52.994336 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6a321e-5734-47d2-8624-76b1d3afb4f0 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:53.057019 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:53.065524 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af6e03-0f30-4402-a69b-12fc3e7cf9b9 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:53.090523 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:53.101501 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6a321e-5734-47d2-8624-76b1d3afb4f0 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:53.102867 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1577d9-6e42-465e-a42c-3bb839fe9dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:53.103635 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1577d9-6e42-465e-a42c-3bb839fe9dbf None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:53.104001 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1577d9-6e42-465e-a42c-3bb839fe9dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:53.104384 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1577d9-6e42-465e-a42c-3bb839fe9dbf None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:53.105282 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1577d9-6e42-465e-a42c-3bb839fe9dbf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:53.105603 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1577d9-6e42-465e-a42c-3bb839fe9dbf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:53.107046 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1577d9-6e42-465e-a42c-3bb839fe9dbf None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:53.121067 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 317/646] 10.222.0.41 () {56 vars in 1148 bytes} [Fri Jun 3 16:02:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:53.121893 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1577d9-6e42-465e-a42c-3bb839fe9dbf None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:53.142798 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b5a06-e2b1-49fe-b328-64328e8016fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:53.143641 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b5a06-e2b1-49fe-b328-64328e8016fb None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:53.143952 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b5a06-e2b1-49fe-b328-64328e8016fb None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:53.144229 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b5a06-e2b1-49fe-b328-64328e8016fb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:53.151993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b5a06-e2b1-49fe-b328-64328e8016fb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:53.153431 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b5a06-e2b1-49fe-b328-64328e8016fb None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:53.169006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-4e6b5a06-e2b1-49fe-b328-64328e8016fb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 03 16:02:53.169006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:53.169006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:53.169006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:53.169006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:53.169006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:53.169006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:53.169006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:53.169006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:53.169006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:53.169006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:53.169006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:53.169006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:02:53.169006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:02:53.169006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:02:53.169006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:02:53.169997 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:53.169997 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:53.169997 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:02:53.169997 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:02:53.169997 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:02:53.169997 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:02:53.169997 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:53.169997 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:53.169997 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:53.169997 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:53.169997 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:53.169997 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:53.169997 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:53.169997 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:53.169997 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:53.169997 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:53.169997 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:53.169997 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:53.170949 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:53.170949 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:53.170949 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:02:53.170949 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:02:53.170949 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:02:53.170949 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:02:53.170949 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:02:53.170949 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:02:53.170949 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 03 16:02:53.170949 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:53.170949 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 330/647] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:02:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:53.172457 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b5a06-e2b1-49fe-b328-64328e8016fb None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:53.240958 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1577d9-6e42-465e-a42c-3bb839fe9dbf None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:53.273502 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b5a06-e2b1-49fe-b328-64328e8016fb None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:53.279049 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f30844-f17e-41d6-aed3-85fc691dce6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:53.279867 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f30844-f17e-41d6-aed3-85fc691dce6b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:53.280151 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f30844-f17e-41d6-aed3-85fc691dce6b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:53.280533 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f30844-f17e-41d6-aed3-85fc691dce6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:53.291914 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f30844-f17e-41d6-aed3-85fc691dce6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:53.293377 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f30844-f17e-41d6-aed3-85fc691dce6b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:53.307865 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a18090-8e46-41dc-84b2-0024500b9861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:53.308532 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a18090-8e46-41dc-84b2-0024500b9861 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:53.308820 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a18090-8e46-41dc-84b2-0024500b9861 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:53.309100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a18090-8e46-41dc-84b2-0024500b9861 None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:53.309843 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a18090-8e46-41dc-84b2-0024500b9861 None admin] RBAC: Adding query filter params (name=service) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:53.310154 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a18090-8e46-41dc-84b2-0024500b9861 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:53.311543 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a18090-8e46-41dc-84b2-0024500b9861 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:53.311806 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-15f30844-f17e-41d6-aed3-85fc691dce6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:53.311806 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:53.311806 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:53.311806 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:53.311806 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:53.311806 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:53.311806 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:53.311806 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:53.311806 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:53.311806 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:53.311806 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:53.311806 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:53.311806 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:53.311806 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:53.311806 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:53.312648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:53.312648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:53.312648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:53.312648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:53.312648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:53.312648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:53.312648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:53.312648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:53.312648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:53.312648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:53.312648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:53.312648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:53.312648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:53.312648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:53.312648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:53.312648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:53.312648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:53.313550 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:53.313550 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:53.313550 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:53.313550 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:53.313550 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 318/648] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:53.314273 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f30844-f17e-41d6-aed3-85fc691dce6b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:53.322373 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:53.324679 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 331/649] 10.222.0.41 () {56 vars in 1103 bytes} [Fri Jun 3 16:02:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:53.329893 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a18090-8e46-41dc-84b2-0024500b9861 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:53.345166 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:53.423937 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f30844-f17e-41d6-aed3-85fc691dce6b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:53.433152 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a18090-8e46-41dc-84b2-0024500b9861 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:53.461247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41460d4-7198-4ec5-a3f1-cbaeaf45f485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:53.462115 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41460d4-7198-4ec5-a3f1-cbaeaf45f485 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:53.462558 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41460d4-7198-4ec5-a3f1-cbaeaf45f485 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:53.462906 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41460d4-7198-4ec5-a3f1-cbaeaf45f485 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:53.463680 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41460d4-7198-4ec5-a3f1-cbaeaf45f485 None admin] RBAC: Adding query filter params (role.id=fe695aedbe8249a28c992a19264bc34b, scope.project.id=c274254e510a468e9acbecc6298301cc, user.id=2fd08ea39ee94b92ac6763608e69f158) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:53.463999 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41460d4-7198-4ec5-a3f1-cbaeaf45f485 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:53.465419 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41460d4-7198-4ec5-a3f1-cbaeaf45f485 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:53.470298 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ddffc4-d246-4c29-a091-0fe85f6edf07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:53.470979 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ddffc4-d246-4c29-a091-0fe85f6edf07 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:53.479379 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ddffc4-d246-4c29-a091-0fe85f6edf07 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:53.479584 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 319/650] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:02:53 2022] GET /identity/v3/role_assignments?user.id=2fd08ea39ee94b92ac6763608e69f158&scope.project.id=c274254e510a468e9acbecc6298301cc&role.id=fe695aedbe8249a28c992a19264bc34b => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:53.479710 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ddffc4-d246-4c29-a091-0fe85f6edf07 None admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:53.479798 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41460d4-7198-4ec5-a3f1-cbaeaf45f485 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:53.479880 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ddffc4-d246-4c29-a091-0fe85f6edf07 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:53.479880 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ddffc4-d246-4c29-a091-0fe85f6edf07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:53.479880 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ddffc4-d246-4c29-a091-0fe85f6edf07 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:53.499782 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 332/651] 10.222.0.41 () {56 vars in 1137 bytes} [Fri Jun 3 16:02:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:53.500745 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ddffc4-d246-4c29-a091-0fe85f6edf07 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:53.563698 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:53.588348 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:53.608489 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41460d4-7198-4ec5-a3f1-cbaeaf45f485 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:53.623277 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ddffc4-d246-4c29-a091-0fe85f6edf07 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:53.658247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3dccc-9c12-425e-9c23-251be1acf261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:53.659127 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3dccc-9c12-425e-9c23-251be1acf261 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:53.659552 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3dccc-9c12-425e-9c23-251be1acf261 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:53.659900 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3dccc-9c12-425e-9c23-251be1acf261 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:53.661018 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb3dccc-9c12-425e-9c23-251be1acf261 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:53.661411 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb3dccc-9c12-425e-9c23-251be1acf261 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:53.662057 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9883e9-0128-487d-a316-e28825a33828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:53.662984 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb3dccc-9c12-425e-9c23-251be1acf261 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:53.663097 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9883e9-0128-487d-a316-e28825a33828 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:53.663266 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9883e9-0128-487d-a316-e28825a33828 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:53.663598 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9883e9-0128-487d-a316-e28825a33828 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:53.675620 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9883e9-0128-487d-a316-e28825a33828 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:53.677091 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9883e9-0128-487d-a316-e28825a33828 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:53.679122 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 320/652] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:53.680659 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3dccc-9c12-425e-9c23-251be1acf261 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:53.701404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-aa9883e9-0128-487d-a316-e28825a33828 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:53.701404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:53.701404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:53.701404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:53.701404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:53.701404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:53.701404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:53.701404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:53.701404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:53.701404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:53.701404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:53.701404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:53.701404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:53.701404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:53.701404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:53.702537 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:53.702537 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:53.702537 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:53.702537 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:53.702537 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:53.702537 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:53.702537 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:53.702537 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:53.702537 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:53.702537 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:53.702537 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:53.702537 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:53.702537 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:53.702537 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:53.702537 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:53.702537 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:53.702537 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:53.703751 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:53.703751 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:53.703751 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:53.703751 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:53.705303 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 333/653] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:53.708735 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9883e9-0128-487d-a316-e28825a33828 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:53.806503 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3dccc-9c12-425e-9c23-251be1acf261 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:53.831759 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9883e9-0128-487d-a316-e28825a33828 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:53.838358 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:53.846419 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b0ea3f-0c92-4d3c-96b7-d0e58938c05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:53.847723 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b0ea3f-0c92-4d3c-96b7-d0e58938c05e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:53.848084 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b0ea3f-0c92-4d3c-96b7-d0e58938c05e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:53.848425 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b0ea3f-0c92-4d3c-96b7-d0e58938c05e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:53.874302 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:53.889273 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e98f0e0-93f1-4832-8c19-bedf87559c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:53.890758 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98f0e0-93f1-4832-8c19-bedf87559c29 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:53.891544 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98f0e0-93f1-4832-8c19-bedf87559c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:53.892002 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98f0e0-93f1-4832-8c19-bedf87559c29 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:53.903241 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e98f0e0-93f1-4832-8c19-bedf87559c29 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:53.905813 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e98f0e0-93f1-4832-8c19-bedf87559c29 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:53.928430 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-2e98f0e0-93f1-4832-8c19-bedf87559c29 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 03 16:02:53.928430 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:53.928430 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:53.928430 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:53.928430 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:53.928430 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:53.928430 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:53.928430 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:53.928430 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:53.928430 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:53.928430 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:53.928430 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:53.928430 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:02:53.928430 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:02:53.928430 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:02:53.928430 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:02:53.929613 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:53.929613 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:53.929613 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:02:53.929613 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:02:53.929613 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:02:53.929613 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:02:53.929613 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:53.929613 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:53.929613 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:53.929613 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:53.929613 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:53.929613 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:53.929613 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:53.929613 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:53.929613 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:53.929613 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:53.929613 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:53.929613 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:53.930776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:53.930776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:53.930776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:02:53.930776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:02:53.930776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:02:53.930776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:02:53.930776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:02:53.930776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:02:53.930776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 03 16:02:53.930776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:53.937609 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 334/654] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:02:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:53.937609 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e98f0e0-93f1-4832-8c19-bedf87559c29 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:53.941471 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b0ea3f-0c92-4d3c-96b7-d0e58938c05e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:53.943028 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b0ea3f-0c92-4d3c-96b7-d0e58938c05e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:54.046623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e98f0e0-93f1-4832-8c19-bedf87559c29 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:54.047909 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 321/655] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:02:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 16:02:54.048991 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b0ea3f-0c92-4d3c-96b7-d0e58938c05e None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:54.080287 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:54.089088 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92ed16-d80e-4fad-807c-35f0eca71cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:54.090005 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92ed16-d80e-4fad-807c-35f0eca71cde None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:54.090497 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92ed16-d80e-4fad-807c-35f0eca71cde None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:54.090990 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92ed16-d80e-4fad-807c-35f0eca71cde None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:54.092420 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d92ed16-d80e-4fad-807c-35f0eca71cde None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:54.092914 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d92ed16-d80e-4fad-807c-35f0eca71cde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:54.094908 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d92ed16-d80e-4fad-807c-35f0eca71cde None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:54.110948 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 335/656] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:54.112052 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92ed16-d80e-4fad-807c-35f0eca71cde None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:54.161444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b0ea3f-0c92-4d3c-96b7-d0e58938c05e None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:54.172144 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:54.207956 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d9621e-f111-459b-a0cc-6b566728eb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:54.208630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d9621e-f111-459b-a0cc-6b566728eb0a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:54.208916 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d9621e-f111-459b-a0cc-6b566728eb0a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:54.209199 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d9621e-f111-459b-a0cc-6b566728eb0a None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:54.210038 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d9621e-f111-459b-a0cc-6b566728eb0a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:54.210350 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d9621e-f111-459b-a0cc-6b566728eb0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:54.211837 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d9621e-f111-459b-a0cc-6b566728eb0a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:54.231462 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 322/657] 10.222.0.41 () {56 vars in 1137 bytes} [Fri Jun 3 16:02:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:54.232323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d9621e-f111-459b-a0cc-6b566728eb0a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:54.241995 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92ed16-d80e-4fad-807c-35f0eca71cde None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:54.280723 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8464cf5a-a220-4975-a749-9fba01149a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:54.281833 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464cf5a-a220-4975-a749-9fba01149a8a None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:54.282126 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464cf5a-a220-4975-a749-9fba01149a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:54.282553 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464cf5a-a220-4975-a749-9fba01149a8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:54.293882 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8464cf5a-a220-4975-a749-9fba01149a8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:54.295318 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8464cf5a-a220-4975-a749-9fba01149a8a None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:54.313474 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-8464cf5a-a220-4975-a749-9fba01149a8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:54.313474 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:54.313474 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:54.313474 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:54.313474 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:54.313474 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:54.313474 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:54.313474 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:54.313474 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:54.313474 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:54.313474 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:54.313474 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:54.313474 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:54.313474 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:54.313474 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:54.314381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:54.314381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:54.314381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:54.314381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:54.314381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:54.314381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:54.314381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:54.314381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:54.314381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:54.314381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:54.314381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:54.314381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:54.314381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:54.314381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:54.314381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:54.314381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:54.314381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:54.315268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:54.315268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:54.315268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:54.315268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:54.315268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 336/658] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:54.317040 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223bd73a-9158-4692-9889-6d8644580e58 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:54.317040 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223bd73a-9158-4692-9889-6d8644580e58 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:54.317186 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223bd73a-9158-4692-9889-6d8644580e58 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:54.318501 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 337/659] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:54.320260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223bd73a-9158-4692-9889-6d8644580e58 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:54.334218 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:54.349332 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d9621e-f111-459b-a0cc-6b566728eb0a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:54.388266 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48d5fa-a4e7-4231-a6fa-9ab592941388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:54.388481 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48d5fa-a4e7-4231-a6fa-9ab592941388 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:54.388756 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48d5fa-a4e7-4231-a6fa-9ab592941388 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:54.388919 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48d5fa-a4e7-4231-a6fa-9ab592941388 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:54.396918 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48d5fa-a4e7-4231-a6fa-9ab592941388 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:54.398387 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48d5fa-a4e7-4231-a6fa-9ab592941388 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:54.413361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-eb48d5fa-a4e7-4231-a6fa-9ab592941388 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 03 16:02:54.413361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:54.413361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:54.413361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:54.413361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:54.413361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:54.413361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:54.413361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:54.413361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:54.413361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:54.413361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:54.413361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:54.413361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:02:54.413361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:02:54.413361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:02:54.413361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:02:54.414303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:54.414303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:54.414303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:02:54.414303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:02:54.414303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:02:54.414303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:02:54.414303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:54.414303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:54.414303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:54.414303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:54.414303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:54.414303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:54.414303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:54.414303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:54.414303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:54.414303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:54.414303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:54.414303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:54.415237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:54.415237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:54.415237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:02:54.415237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:02:54.415237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:02:54.415237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:02:54.415237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:02:54.415237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:02:54.415237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 03 16:02:54.415237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:54.415237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 323/660] 10.222.0.41 () {56 vars in 1102 bytes} [Fri Jun 3 16:02:54 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:54.415895 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48d5fa-a4e7-4231-a6fa-9ab592941388 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:54.435653 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:54.444203 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223bd73a-9158-4692-9889-6d8644580e58 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:54.483138 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ff9e5c-f96f-49ba-8fd3-bf9bba1758be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:54.483823 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff9e5c-f96f-49ba-8fd3-bf9bba1758be None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:54.484256 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff9e5c-f96f-49ba-8fd3-bf9bba1758be None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:54.484541 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff9e5c-f96f-49ba-8fd3-bf9bba1758be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:54.521374 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48d5fa-a4e7-4231-a6fa-9ab592941388 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:54.557487 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7c254d-1abe-4f9d-9a94-478055316912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:54.558074 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c254d-1abe-4f9d-9a94-478055316912 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:54.558358 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c254d-1abe-4f9d-9a94-478055316912 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:54.558636 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c254d-1abe-4f9d-9a94-478055316912 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:54.559384 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7c254d-1abe-4f9d-9a94-478055316912 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:54.559696 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7c254d-1abe-4f9d-9a94-478055316912 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:54.559884 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ff9e5c-f96f-49ba-8fd3-bf9bba1758be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:54.561061 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7c254d-1abe-4f9d-9a94-478055316912 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:54.561316 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ff9e5c-f96f-49ba-8fd3-bf9bba1758be None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:54.577387 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 324/661] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:54.577387 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f8e54-9763-4507-8cb5-938cdaca0b04 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:54.577387 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f8e54-9763-4507-8cb5-938cdaca0b04 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:54.577387 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f8e54-9763-4507-8cb5-938cdaca0b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:54.592696 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-777f8e54-9763-4507-8cb5-938cdaca0b04 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:54.593817 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-777f8e54-9763-4507-8cb5-938cdaca0b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:54.595878 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:54.665229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 325/662] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:54.665944 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f8e54-9763-4507-8cb5-938cdaca0b04 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:54.683327 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:54.691567 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 338/663] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:02:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 16:02:54.692311 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ff9e5c-f96f-49ba-8fd3-bf9bba1758be None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:54.790850 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f8e54-9763-4507-8cb5-938cdaca0b04 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:54.796413 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ff9e5c-f96f-49ba-8fd3-bf9bba1758be None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:54.828090 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d39f4-ee07-4680-bade-0ae2fe8b753e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:54.828765 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d39f4-ee07-4680-bade-0ae2fe8b753e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:54.829055 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d39f4-ee07-4680-bade-0ae2fe8b753e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:54.829335 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d39f4-ee07-4680-bade-0ae2fe8b753e None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:54.830181 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8d39f4-ee07-4680-bade-0ae2fe8b753e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:54.830495 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8d39f4-ee07-4680-bade-0ae2fe8b753e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:54.832004 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8d39f4-ee07-4680-bade-0ae2fe8b753e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:54.835796 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e16b0-d87d-4a5e-a3fb-f46b49531d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:54.836770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e16b0-d87d-4a5e-a3fb-f46b49531d80 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:54.837199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e16b0-d87d-4a5e-a3fb-f46b49531d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:54.837619 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e16b0-d87d-4a5e-a3fb-f46b49531d80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:54.840265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:54.845479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8e16b0-d87d-4a5e-a3fb-f46b49531d80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:54.847402 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8e16b0-d87d-4a5e-a3fb-f46b49531d80 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:54.852904 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 326/664] 10.222.0.41 () {56 vars in 1139 bytes} [Fri Jun 3 16:02:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:54.853748 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d39f4-ee07-4680-bade-0ae2fe8b753e None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:54.865748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-0a8e16b0-d87d-4a5e-a3fb-f46b49531d80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:02:54.865748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:54.865748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:54.865748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:54.865748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:54.865748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:54.865748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:54.865748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:54.865748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:54.865748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:54.865748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:54.865748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:54.865748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:02:54.865748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:02:54.865748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:02:54.866678 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:02:54.866678 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:54.866678 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:54.866678 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:54.866678 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:54.866678 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:54.866678 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:54.866678 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:54.866678 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:54.866678 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:54.866678 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:54.866678 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:54.866678 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:54.866678 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:54.866678 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:54.866678 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:54.866678 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:54.868392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:02:54.868392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:02:54.868392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:02:54.868392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:02:54.868392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:02:54.868392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:02:54.868392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:02:54.868392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:54.868392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 339/665] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:02:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:54.868392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e16b0-d87d-4a5e-a3fb-f46b49531d80 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:54.934164 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:54.964564 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d39f4-ee07-4680-bade-0ae2fe8b753e None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:54.982544 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e16b0-d87d-4a5e-a3fb-f46b49531d80 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:55.004005 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9080688-38cc-4e8e-b9b7-56af2485fb76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:55.004926 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9080688-38cc-4e8e-b9b7-56af2485fb76 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:55.005255 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9080688-38cc-4e8e-b9b7-56af2485fb76 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:55.005643 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9080688-38cc-4e8e-b9b7-56af2485fb76 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:55.016892 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9080688-38cc-4e8e-b9b7-56af2485fb76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:55.024676 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-b9080688-38cc-4e8e-b9b7-56af2485fb76 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:02:55.024676 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:55.024676 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:55.024676 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:55.024676 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:55.024676 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:55.024676 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:55.024676 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:55.024676 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:55.024676 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:55.024676 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:55.024676 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:55.024676 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:55.024676 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:55.024676 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:55.024676 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:55.025588 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:55.025588 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:55.025588 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:55.025588 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:55.025588 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:55.025588 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:55.025588 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:55.025588 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:55.025588 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:55.025588 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:55.025588 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:55.025588 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:55.025588 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:55.025588 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:55.025588 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:55.025588 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:55.026406 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:55.026406 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:55.026406 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:55.026406 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:55.026406 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:55.026406 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:55.026406 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:55.026406 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:55.026406 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:55.026406 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:55.026406 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:02:55.026406 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:55.030367 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd95dc8-3181-4eb4-ad87-4156c48051fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:55.030918 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 327/666] 10.222.0.41 () {56 vars in 1102 bytes} [Fri Jun 3 16:02:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:55.031423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd95dc8-3181-4eb4-ad87-4156c48051fa None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:55.031709 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd95dc8-3181-4eb4-ad87-4156c48051fa None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:55.031801 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9080688-38cc-4e8e-b9b7-56af2485fb76 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:55.031983 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd95dc8-3181-4eb4-ad87-4156c48051fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:55.043368 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd95dc8-3181-4eb4-ad87-4156c48051fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:55.044787 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd95dc8-3181-4eb4-ad87-4156c48051fa None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:55.062114 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-bdd95dc8-3181-4eb4-ad87-4156c48051fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:55.062114 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:55.062114 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:55.062114 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:55.062114 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:55.062114 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:55.062114 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:55.062114 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:55.062114 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:55.062114 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:55.062114 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:55.062114 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:55.062114 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:55.062114 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:55.062114 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:55.062991 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:55.062991 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:55.062991 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:55.062991 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:55.062991 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:55.062991 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:55.062991 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:55.062991 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:55.062991 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:55.062991 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:55.062991 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:55.062991 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:55.062991 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:55.062991 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:55.062991 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:55.062991 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:55.062991 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:55.063915 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:55.063915 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:55.063915 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:55.063915 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:55.063915 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 340/667] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:55.064613 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd95dc8-3181-4eb4-ad87-4156c48051fa None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:55.094770 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:55.141737 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9080688-38cc-4e8e-b9b7-56af2485fb76 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:55.175345 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd95dc8-3181-4eb4-ad87-4156c48051fa None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:55.178178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e78d198-27c6-493c-a520-c733afc79a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:55.178178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78d198-27c6-493c-a520-c733afc79a78 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:55.178467 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78d198-27c6-493c-a520-c733afc79a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:55.178712 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78d198-27c6-493c-a520-c733afc79a78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:55.185146 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:55.214388 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605854a-86d9-466e-8f75-588b24ec3141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:55.215041 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605854a-86d9-466e-8f75-588b24ec3141 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:55.215356 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605854a-86d9-466e-8f75-588b24ec3141 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:55.215634 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605854a-86d9-466e-8f75-588b24ec3141 None admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:55.216418 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9605854a-86d9-466e-8f75-588b24ec3141 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:55.216726 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9605854a-86d9-466e-8f75-588b24ec3141 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:55.218178 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9605854a-86d9-466e-8f75-588b24ec3141 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:55.231248 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 341/668] 10.222.0.41 () {56 vars in 1148 bytes} [Fri Jun 3 16:02:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:55.232088 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605854a-86d9-466e-8f75-588b24ec3141 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:55.264887 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e78d198-27c6-493c-a520-c733afc79a78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:55.266348 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e78d198-27c6-493c-a520-c733afc79a78 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:55.325121 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:55.341543 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605854a-86d9-466e-8f75-588b24ec3141 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:55.375062 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 328/669] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:02:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 16:02:55.375907 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e78d198-27c6-493c-a520-c733afc79a78 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:55.382428 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695d48f-4a6a-4c68-abbf-4580a775c992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:55.383108 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695d48f-4a6a-4c68-abbf-4580a775c992 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:55.383425 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695d48f-4a6a-4c68-abbf-4580a775c992 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:55.383713 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695d48f-4a6a-4c68-abbf-4580a775c992 None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:55.384466 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d695d48f-4a6a-4c68-abbf-4580a775c992 None admin] RBAC: Adding query filter params (name=service) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:55.384776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d695d48f-4a6a-4c68-abbf-4580a775c992 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:55.386131 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d695d48f-4a6a-4c68-abbf-4580a775c992 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:55.399753 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 342/670] 10.222.0.41 () {56 vars in 1103 bytes} [Fri Jun 3 16:02:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:55.399753 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695d48f-4a6a-4c68-abbf-4580a775c992 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:55.429615 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:55.485076 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e78d198-27c6-493c-a520-c733afc79a78 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:55.505202 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695d48f-4a6a-4c68-abbf-4580a775c992 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:55.521816 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc55ee1-26f0-4630-ae13-1d23db7717e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:55.522587 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc55ee1-26f0-4630-ae13-1d23db7717e5 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:55.522873 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc55ee1-26f0-4630-ae13-1d23db7717e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:55.523150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc55ee1-26f0-4630-ae13-1d23db7717e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:55.524007 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc55ee1-26f0-4630-ae13-1d23db7717e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:55.524314 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc55ee1-26f0-4630-ae13-1d23db7717e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:55.525804 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc55ee1-26f0-4630-ae13-1d23db7717e5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:55.541106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d303e0-1211-4f59-8e0a-0a00ca09735a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:55.541756 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d303e0-1211-4f59-8e0a-0a00ca09735a None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:55.542035 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d303e0-1211-4f59-8e0a-0a00ca09735a None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:55.542306 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d303e0-1211-4f59-8e0a-0a00ca09735a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:55.542842 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 329/671] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:55.543132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d303e0-1211-4f59-8e0a-0a00ca09735a None admin] RBAC: Adding query filter params (role.id=fe695aedbe8249a28c992a19264bc34b, scope.project.id=c274254e510a468e9acbecc6298301cc, user.id=62a3bbb9720049b29eb37b0dd88bbad7) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:55.543473 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d303e0-1211-4f59-8e0a-0a00ca09735a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:55.543715 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc55ee1-26f0-4630-ae13-1d23db7717e5 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:55.544893 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d303e0-1211-4f59-8e0a-0a00ca09735a None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:55.559730 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 343/672] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:02:55 2022] GET /identity/v3/role_assignments?user.id=62a3bbb9720049b29eb37b0dd88bbad7&scope.project.id=c274254e510a468e9acbecc6298301cc&role.id=fe695aedbe8249a28c992a19264bc34b => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:55.559730 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d303e0-1211-4f59-8e0a-0a00ca09735a None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:55.563144 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:55.679674 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:55.684203 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc55ee1-26f0-4630-ae13-1d23db7717e5 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:55.688706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d303e0-1211-4f59-8e0a-0a00ca09735a None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:55.727680 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae88b1e-8bf3-47eb-b52a-3e04aedf9863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:55.728684 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae88b1e-8bf3-47eb-b52a-3e04aedf9863 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:55.729071 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae88b1e-8bf3-47eb-b52a-3e04aedf9863 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:55.729448 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae88b1e-8bf3-47eb-b52a-3e04aedf9863 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:55.729894 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d03f13-e657-4958-96ef-a944ec4c984f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:55.730866 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d03f13-e657-4958-96ef-a944ec4c984f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:55.731150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d03f13-e657-4958-96ef-a944ec4c984f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:55.731460 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d03f13-e657-4958-96ef-a944ec4c984f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:55.742698 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae88b1e-8bf3-47eb-b52a-3e04aedf9863 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:55.744370 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae88b1e-8bf3-47eb-b52a-3e04aedf9863 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:55.745195 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d03f13-e657-4958-96ef-a944ec4c984f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:55.746839 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d03f13-e657-4958-96ef-a944ec4c984f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:55.765193 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-6ae88b1e-8bf3-47eb-b52a-3e04aedf9863 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:55.765193 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:55.765193 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:55.765193 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:55.765193 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:55.765193 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:55.765193 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:55.765193 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:55.765193 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:55.765193 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:55.765193 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:55.765193 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:55.765193 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:55.765193 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:55.765193 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:55.767337 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:55.767337 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:55.767337 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:55.767337 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:55.767337 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:55.767337 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:55.767337 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:55.767337 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:55.767337 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:55.767337 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:55.767337 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:55.767337 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:55.767337 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:55.767337 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:55.767337 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:55.767337 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:55.767337 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:55.770008 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:55.770008 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:55.770008 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:55.770008 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:55.770496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-c3d03f13-e657-4958-96ef-a944ec4c984f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:55.770496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:55.770496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:55.770496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:55.770496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:55.770496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:55.770496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:55.770496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:55.770496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:55.770496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:55.770496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:55.770496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:55.770496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:55.770496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:55.770496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:55.773345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:55.773345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:55.773345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:55.773345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:55.773345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:55.773345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:55.773345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:55.773345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:55.773345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:55.773345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:55.773345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:55.773345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:55.773345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:55.773345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:55.773345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:55.773345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:55.773345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:55.775436 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:55.775436 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:55.775436 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:55.775436 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:55.775938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 344/673] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:55.776075 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 330/674] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:55.776227 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae88b1e-8bf3-47eb-b52a-3e04aedf9863 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:55.776366 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d03f13-e657-4958-96ef-a944ec4c984f None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:55.853943 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:55.900417 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d03f13-e657-4958-96ef-a944ec4c984f None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:55.902891 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae88b1e-8bf3-47eb-b52a-3e04aedf9863 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:55.941479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6550d75-e495-4754-8ed5-b317a1d968c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:55.942250 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6550d75-e495-4754-8ed5-b317a1d968c8 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:55.942552 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6550d75-e495-4754-8ed5-b317a1d968c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:55.942852 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6550d75-e495-4754-8ed5-b317a1d968c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:55.947097 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abff2cd-546c-4246-98b2-1f564945d884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:55.947943 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abff2cd-546c-4246-98b2-1f564945d884 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:55.948369 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abff2cd-546c-4246-98b2-1f564945d884 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:55.948806 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abff2cd-546c-4246-98b2-1f564945d884 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:55.949954 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abff2cd-546c-4246-98b2-1f564945d884 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:55.950145 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6550d75-e495-4754-8ed5-b317a1d968c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:55.950561 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abff2cd-546c-4246-98b2-1f564945d884 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:55.951745 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6550d75-e495-4754-8ed5-b317a1d968c8 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:55.952275 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abff2cd-546c-4246-98b2-1f564945d884 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:55.966645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 331/675] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:55.966793 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-d6550d75-e495-4754-8ed5-b317a1d968c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:02:55.966793 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:55.966793 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:55.966793 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:55.966793 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:55.966793 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:55.966793 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:55.966793 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:55.966793 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:55.966793 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:55.966793 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:55.966793 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:55.966793 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:02:55.966793 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:02:55.966793 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:02:55.967715 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:02:55.967715 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:55.967715 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:55.967715 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:55.967715 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:55.967715 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:55.967715 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:55.967715 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:55.967715 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:55.967715 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:55.967715 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:55.967715 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:55.967715 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:55.967715 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:55.967715 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:55.967715 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:55.967715 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:55.968577 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:02:55.968577 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:02:55.968577 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:02:55.968577 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:02:55.968577 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:02:55.968577 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:02:55.968577 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:02:55.968577 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:55.969919 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abff2cd-546c-4246-98b2-1f564945d884 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:55.972603 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 345/676] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:02:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:55.974667 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba527089-7e73-4bd5-93a1-c31ba44caa11 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:55.974667 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba527089-7e73-4bd5-93a1-c31ba44caa11 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:55.974667 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba527089-7e73-4bd5-93a1-c31ba44caa11 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:55.976077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 346/677] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:55.976689 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba527089-7e73-4bd5-93a1-c31ba44caa11 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:55.989136 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:56.081236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abff2cd-546c-4246-98b2-1f564945d884 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:56.089761 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba527089-7e73-4bd5-93a1-c31ba44caa11 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:56.108630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:56.117556 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9456e68-b156-4e28-a312-09575ee5352b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:56.118226 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9456e68-b156-4e28-a312-09575ee5352b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:56.118514 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9456e68-b156-4e28-a312-09575ee5352b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:56.118790 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9456e68-b156-4e28-a312-09575ee5352b None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:56.119545 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9456e68-b156-4e28-a312-09575ee5352b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:56.119855 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9456e68-b156-4e28-a312-09575ee5352b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:56.121229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9456e68-b156-4e28-a312-09575ee5352b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:56.130742 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f64494-6479-4150-8be5-2ccd285a24f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:56.131795 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f64494-6479-4150-8be5-2ccd285a24f3 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:56.132273 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f64494-6479-4150-8be5-2ccd285a24f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:56.132420 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 332/678] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:56.132841 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f64494-6479-4150-8be5-2ccd285a24f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:56.133223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9456e68-b156-4e28-a312-09575ee5352b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:56.143999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f64494-6479-4150-8be5-2ccd285a24f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:56.145655 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f64494-6479-4150-8be5-2ccd285a24f3 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:56.162999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-18f64494-6479-4150-8be5-2ccd285a24f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:02:56.162999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:56.162999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:56.162999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:56.162999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:56.162999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:56.162999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:56.162999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:56.162999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:56.162999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:56.162999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:56.162999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:56.162999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:02:56.162999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:02:56.162999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:02:56.163977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:02:56.163977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:56.163977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:56.163977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:56.163977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:56.163977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:56.163977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:56.163977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:56.163977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:56.163977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:56.163977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:56.163977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:56.163977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:56.163977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:56.163977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:56.163977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:56.163977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:56.169669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:02:56.169669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:02:56.169669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:02:56.169669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:02:56.169669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:02:56.169669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:02:56.169669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:02:56.169669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:56.169669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 347/679] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:02:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:56.169669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a4d5c-1e78-4ed4-919b-5f13d646de23 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:56.169669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a4d5c-1e78-4ed4-919b-5f13d646de23 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:56.170284 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a4d5c-1e78-4ed4-919b-5f13d646de23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:56.188245 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-e50a4d5c-1e78-4ed4-919b-5f13d646de23 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:56.189455 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-e50a4d5c-1e78-4ed4-919b-5f13d646de23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:56.240241 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9456e68-b156-4e28-a312-09575ee5352b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:56.266353 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:56.279396 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 348/680] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:56.279396 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a4d5c-1e78-4ed4-919b-5f13d646de23 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:56.288019 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224acd8-0cbd-4890-86be-6d51703e8c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:56.288931 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224acd8-0cbd-4890-86be-6d51703e8c07 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:56.289353 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224acd8-0cbd-4890-86be-6d51703e8c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:56.289805 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224acd8-0cbd-4890-86be-6d51703e8c07 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:56.291457 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224acd8-0cbd-4890-86be-6d51703e8c07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:56.292305 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224acd8-0cbd-4890-86be-6d51703e8c07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:56.295066 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224acd8-0cbd-4890-86be-6d51703e8c07 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:56.310358 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 333/681] 10.222.0.41 () {56 vars in 1148 bytes} [Fri Jun 3 16:02:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:56.311904 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224acd8-0cbd-4890-86be-6d51703e8c07 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:56.371495 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:56.381506 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a4d5c-1e78-4ed4-919b-5f13d646de23 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:56.418595 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3065db-4b6f-4ae5-8c16-e55c91cecb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:56.419700 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3065db-4b6f-4ae5-8c16-e55c91cecb7a None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:56.419985 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3065db-4b6f-4ae5-8c16-e55c91cecb7a None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:56.420256 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3065db-4b6f-4ae5-8c16-e55c91cecb7a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:56.428585 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3065db-4b6f-4ae5-8c16-e55c91cecb7a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:56.430018 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3065db-4b6f-4ae5-8c16-e55c91cecb7a None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:56.434422 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224acd8-0cbd-4890-86be-6d51703e8c07 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:56.446546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-fa3065db-4b6f-4ae5-8c16-e55c91cecb7a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 03 16:02:56.446546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:56.446546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:56.446546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:56.446546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:56.446546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:56.446546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:56.446546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:56.446546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:56.446546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:56.446546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:56.446546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:56.446546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:02:56.446546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:02:56.446546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:02:56.446546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:02:56.447619 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:56.447619 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:56.447619 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:02:56.447619 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:02:56.447619 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:02:56.447619 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:02:56.447619 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:56.447619 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:56.447619 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:56.447619 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:56.447619 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:56.447619 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:56.447619 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:56.447619 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:56.447619 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:56.447619 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:56.447619 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:56.447619 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:56.448563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:56.448563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:56.448563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:02:56.448563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:02:56.448563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:02:56.448563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:02:56.448563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:02:56.448563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:02:56.448563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 03 16:02:56.448563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:56.448563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 349/682] 10.222.0.41 () {56 vars in 1096 bytes} [Fri Jun 3 16:02:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:56.448563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3065db-4b6f-4ae5-8c16-e55c91cecb7a None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:56.475383 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b3c48-76e6-41d4-8afd-a3c6625eac58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:56.476052 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b3c48-76e6-41d4-8afd-a3c6625eac58 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:56.476337 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b3c48-76e6-41d4-8afd-a3c6625eac58 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:56.476618 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b3c48-76e6-41d4-8afd-a3c6625eac58 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:56.477409 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023b3c48-76e6-41d4-8afd-a3c6625eac58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:56.477722 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023b3c48-76e6-41d4-8afd-a3c6625eac58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:56.479166 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023b3c48-76e6-41d4-8afd-a3c6625eac58 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:56.491938 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 334/683] 10.222.0.41 () {56 vars in 1148 bytes} [Fri Jun 3 16:02:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:56.492667 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b3c48-76e6-41d4-8afd-a3c6625eac58 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:56.513521 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:56.558642 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3065db-4b6f-4ae5-8c16-e55c91cecb7a None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:56.598181 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a8a902-c5ac-4353-a1b8-763076a1c49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:56.598983 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a8a902-c5ac-4353-a1b8-763076a1c49e None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:56.599258 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a8a902-c5ac-4353-a1b8-763076a1c49e None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:56.599567 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a8a902-c5ac-4353-a1b8-763076a1c49e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:56.599879 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b3c48-76e6-41d4-8afd-a3c6625eac58 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:56.609533 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:56.610390 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a8a902-c5ac-4353-a1b8-763076a1c49e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:02:56.618706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-d4a8a902-c5ac-4353-a1b8-763076a1c49e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:02:56.618706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:56.618706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:56.618706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:56.618706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:56.618706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:56.618706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:56.618706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:56.618706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:56.618706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:56.618706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:56.618706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:56.618706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:02:56.618706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:02:56.618706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:02:56.618706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:02:56.620113 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:02:56.620113 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:02:56.620113 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:02:56.620113 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:02:56.620113 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:56.620113 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:56.620113 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:56.620113 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:56.620113 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:56.620113 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:56.620113 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:56.620113 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:56.620113 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:56.620113 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:56.620113 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:56.620113 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:56.620943 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:56.620943 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:56.620943 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:56.620943 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:56.620943 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:02:56.620943 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:02:56.620943 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:02:56.620943 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:02:56.620943 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:02:56.620943 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:02:56.620943 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:02:56.620943 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:56.624757 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 350/684] 10.222.0.41 () {56 vars in 1096 bytes} [Fri Jun 3 16:02:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:56.625577 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a8a902-c5ac-4353-a1b8-763076a1c49e None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:56.638578 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f80e8-a67f-489d-8fd7-1dc71039aa60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:56.639236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f80e8-a67f-489d-8fd7-1dc71039aa60 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:56.639549 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f80e8-a67f-489d-8fd7-1dc71039aa60 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:56.639835 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f80e8-a67f-489d-8fd7-1dc71039aa60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:56.640660 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9f80e8-a67f-489d-8fd7-1dc71039aa60 None admin] RBAC: Adding query filter params (role.id=fe695aedbe8249a28c992a19264bc34b, scope.project.id=c274254e510a468e9acbecc6298301cc, user.id=bbaf66d6e1f54047917cc615546bbd29) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:56.640976 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9f80e8-a67f-489d-8fd7-1dc71039aa60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:56.642401 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9f80e8-a67f-489d-8fd7-1dc71039aa60 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:56.648856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 335/685] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:02:56 2022] GET /identity/v3/role_assignments?user.id=bbaf66d6e1f54047917cc615546bbd29&scope.project.id=c274254e510a468e9acbecc6298301cc&role.id=fe695aedbe8249a28c992a19264bc34b => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:56.649694 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f80e8-a67f-489d-8fd7-1dc71039aa60 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:56.757962 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a8a902-c5ac-4353-a1b8-763076a1c49e None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:56.777335 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:56.791332 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f80e8-a67f-489d-8fd7-1dc71039aa60 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:56.798802 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58462dc-9d96-4d55-a83d-739ef18939b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:56.799587 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58462dc-9d96-4d55-a83d-739ef18939b9 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:56.800457 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58462dc-9d96-4d55-a83d-739ef18939b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:56.800748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58462dc-9d96-4d55-a83d-739ef18939b9 None admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:56.801627 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58462dc-9d96-4d55-a83d-739ef18939b9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:56.801944 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58462dc-9d96-4d55-a83d-739ef18939b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:56.803480 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58462dc-9d96-4d55-a83d-739ef18939b9 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:56.824483 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 351/686] 10.222.0.41 () {56 vars in 1135 bytes} [Fri Jun 3 16:02:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:56.825437 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58462dc-9d96-4d55-a83d-739ef18939b9 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:56.836983 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5d953-07b9-4b6c-90fb-4fb2a263c5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:56.838036 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5d953-07b9-4b6c-90fb-4fb2a263c5db None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:56.838527 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5d953-07b9-4b6c-90fb-4fb2a263c5db None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:56.839046 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5d953-07b9-4b6c-90fb-4fb2a263c5db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:56.840012 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e5d953-07b9-4b6c-90fb-4fb2a263c5db None admin] RBAC: Adding query filter params (role.id=fe695aedbe8249a28c992a19264bc34b, scope.project.id=c274254e510a468e9acbecc6298301cc, user.id=aeeaad0680074179b3d29abcf0d51515) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:56.840406 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e5d953-07b9-4b6c-90fb-4fb2a263c5db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:56.841981 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e5d953-07b9-4b6c-90fb-4fb2a263c5db None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:56.849476 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 336/687] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:02:56 2022] GET /identity/v3/role_assignments?user.id=aeeaad0680074179b3d29abcf0d51515&scope.project.id=c274254e510a468e9acbecc6298301cc&role.id=fe695aedbe8249a28c992a19264bc34b => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:56.850484 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5d953-07b9-4b6c-90fb-4fb2a263c5db None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:56.896880 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:56.978408 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58462dc-9d96-4d55-a83d-739ef18939b9 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:56.979895 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5d953-07b9-4b6c-90fb-4fb2a263c5db None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:57.028168 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8d6e9-2218-4863-a421-5c0845ac8575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:57.029505 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8d6e9-2218-4863-a421-5c0845ac8575 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:57.030000 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8d6e9-2218-4863-a421-5c0845ac8575 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:57.030481 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8d6e9-2218-4863-a421-5c0845ac8575 None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:57.031538 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8d6e9-2218-4863-a421-5c0845ac8575 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:57.032078 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8d6e9-2218-4863-a421-5c0845ac8575 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:57.033699 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8d6e9-2218-4863-a421-5c0845ac8575 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:57.036821 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54461631-b040-46e5-bbe3-d6ed9f14cae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:57.037498 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54461631-b040-46e5-bbe3-d6ed9f14cae6 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:57.037786 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54461631-b040-46e5-bbe3-d6ed9f14cae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:57.038109 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54461631-b040-46e5-bbe3-d6ed9f14cae6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:57.047258 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 352/688] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:02:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:57.052444 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8d6e9-2218-4863-a421-5c0845ac8575 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:57.093731 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:57.121790 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54461631-b040-46e5-bbe3-d6ed9f14cae6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:57.123666 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54461631-b040-46e5-bbe3-d6ed9f14cae6 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:57.169306 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:57.175595 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8d6e9-2218-4863-a421-5c0845ac8575 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:57.217107 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9975c3a-26ac-49c0-9e37-79f72a1e3ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:57.217941 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9975c3a-26ac-49c0-9e37-79f72a1e3ec2 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:57.218277 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9975c3a-26ac-49c0-9e37-79f72a1e3ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:57.218500 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9975c3a-26ac-49c0-9e37-79f72a1e3ec2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:57.232635 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9975c3a-26ac-49c0-9e37-79f72a1e3ec2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:57.234202 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9975c3a-26ac-49c0-9e37-79f72a1e3ec2 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:57.253839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-e9975c3a-26ac-49c0-9e37-79f72a1e3ec2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:57.253839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:57.253839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:57.253839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:57.253839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:57.253839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:57.253839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:57.253839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:57.253839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:57.253839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:57.253839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:57.253839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:57.253839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:57.253839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:57.253839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:57.254708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:57.254708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:57.254708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:57.254708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:57.254708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:57.254708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:57.254708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:57.254708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:57.254708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:57.254708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:57.254708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:57.254708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:57.254708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:57.254708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:57.254708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:57.254708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:57.254708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:57.259730 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:57.259730 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:57.259730 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:57.259730 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:57.259730 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 353/689] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:02:57.262722 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9975c3a-26ac-49c0-9e37-79f72a1e3ec2 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:57.279350 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 337/690] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:02:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 16:02:57.284353 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54461631-b040-46e5-bbe3-d6ed9f14cae6 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:57.363724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:57.377651 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9975c3a-26ac-49c0-9e37-79f72a1e3ec2 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:57.406901 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54461631-b040-46e5-bbe3-d6ed9f14cae6 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:57.414011 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e726f13-bf92-496b-a79e-272daa1580a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:57.414682 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e726f13-bf92-496b-a79e-272daa1580a2 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:57.415015 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e726f13-bf92-496b-a79e-272daa1580a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:57.415319 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e726f13-bf92-496b-a79e-272daa1580a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:57.416049 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e726f13-bf92-496b-a79e-272daa1580a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:57.416357 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e726f13-bf92-496b-a79e-272daa1580a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:57.423317 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e726f13-bf92-496b-a79e-272daa1580a2 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:57.437960 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 354/691] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:57.444515 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e726f13-bf92-496b-a79e-272daa1580a2 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:57.465310 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a653170-e55d-45cc-b3cd-c1827700e4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:57.466609 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a653170-e55d-45cc-b3cd-c1827700e4b5 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:57.467163 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a653170-e55d-45cc-b3cd-c1827700e4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:57.467667 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a653170-e55d-45cc-b3cd-c1827700e4b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:57.470992 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:57.497510 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a653170-e55d-45cc-b3cd-c1827700e4b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:57.499433 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a653170-e55d-45cc-b3cd-c1827700e4b5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:57.538989 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-9a653170-e55d-45cc-b3cd-c1827700e4b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:57.538989 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:57.538989 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:57.538989 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:57.538989 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:57.538989 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:57.538989 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:57.538989 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:57.538989 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:57.538989 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:57.538989 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:57.538989 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:57.538989 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:57.538989 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:57.538989 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:57.539943 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:57.539943 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:57.539943 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:57.539943 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:57.539943 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:57.539943 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:57.539943 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:57.539943 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:57.539943 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:57.539943 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:57.539943 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:57.539943 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:57.539943 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:57.539943 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:57.539943 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:57.539943 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:57.539943 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:57.540844 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:57.540844 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:57.540844 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:57.540844 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:57.542952 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 338/692] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:02:57.556677 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a653170-e55d-45cc-b3cd-c1827700e4b5 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:57.557932 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e726f13-bf92-496b-a79e-272daa1580a2 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:57.595945 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d9c13-a7eb-4e33-a0b0-9f4f02d4d3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:57.597344 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d9c13-a7eb-4e33-a0b0-9f4f02d4d3b1 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:57.597634 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d9c13-a7eb-4e33-a0b0-9f4f02d4d3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:57.597954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d9c13-a7eb-4e33-a0b0-9f4f02d4d3b1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:57.606327 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988d9c13-a7eb-4e33-a0b0-9f4f02d4d3b1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:57.607810 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988d9c13-a7eb-4e33-a0b0-9f4f02d4d3b1 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:57.624825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-988d9c13-a7eb-4e33-a0b0-9f4f02d4d3b1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 03 16:02:57.624825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:57.624825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:57.624825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:57.624825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:57.624825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:57.624825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:57.624825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:57.624825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:57.624825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:57.624825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:57.624825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:57.624825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:02:57.624825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:02:57.624825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:02:57.624825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:02:57.625853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:57.625853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:57.625853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:02:57.625853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:02:57.625853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:02:57.625853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:02:57.625853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:57.625853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:57.625853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:57.625853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:57.625853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:57.625853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:57.625853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:57.625853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:57.625853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:57.625853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:57.625853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:57.625853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:57.626800 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:57.626800 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:57.626800 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:02:57.626800 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:02:57.626800 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:02:57.626800 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:02:57.626800 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:02:57.626800 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:02:57.626800 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 03 16:02:57.626800 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:57.626800 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 355/693] 10.222.0.41 () {56 vars in 1093 bytes} [Fri Jun 3 16:02:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:02:57.634318 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d9c13-a7eb-4e33-a0b0-9f4f02d4d3b1 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:57.653403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:57.774764 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a653170-e55d-45cc-b3cd-c1827700e4b5 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:57.804826 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d9c13-a7eb-4e33-a0b0-9f4f02d4d3b1 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:57.844551 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08059b3-cd04-45aa-bbcc-8fe0d7eee46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:57.847093 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08059b3-cd04-45aa-bbcc-8fe0d7eee46b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:57.847770 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08059b3-cd04-45aa-bbcc-8fe0d7eee46b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:57.849264 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08059b3-cd04-45aa-bbcc-8fe0d7eee46b None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:57.850388 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08059b3-cd04-45aa-bbcc-8fe0d7eee46b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:57.851688 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08059b3-cd04-45aa-bbcc-8fe0d7eee46b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:57.855259 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08059b3-cd04-45aa-bbcc-8fe0d7eee46b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:57.870669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb91fb-5316-421a-8616-a5bfbbb9efab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:57.873173 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb91fb-5316-421a-8616-a5bfbbb9efab None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:57.873687 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb91fb-5316-421a-8616-a5bfbbb9efab None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:57.874166 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb91fb-5316-421a-8616-a5bfbbb9efab None admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:57.875375 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb91fb-5316-421a-8616-a5bfbbb9efab None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:57.875969 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb91fb-5316-421a-8616-a5bfbbb9efab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:57.878437 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb91fb-5316-421a-8616-a5bfbbb9efab None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:57.881936 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 339/694] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:02:57.888371 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08059b3-cd04-45aa-bbcc-8fe0d7eee46b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:57.911106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 356/695] 10.222.0.41 () {56 vars in 1133 bytes} [Fri Jun 3 16:02:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:57.917031 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb91fb-5316-421a-8616-a5bfbbb9efab None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:57.976778 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:58.029576 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:58.070982 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08059b3-cd04-45aa-bbcc-8fe0d7eee46b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:58.076794 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb91fb-5316-421a-8616-a5bfbbb9efab None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:58.113849 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19393ed0-0777-4905-bfb2-a5619734061d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:58.114669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19393ed0-0777-4905-bfb2-a5619734061d None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:58.115094 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19393ed0-0777-4905-bfb2-a5619734061d None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:58.115526 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19393ed0-0777-4905-bfb2-a5619734061d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:58.118763 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5104074-0db2-454d-a5b6-6070ca6c91fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:58.120174 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5104074-0db2-454d-a5b6-6070ca6c91fd None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:58.120455 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5104074-0db2-454d-a5b6-6070ca6c91fd None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:58.120728 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5104074-0db2-454d-a5b6-6070ca6c91fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:58.132000 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5104074-0db2-454d-a5b6-6070ca6c91fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:58.134620 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5104074-0db2-454d-a5b6-6070ca6c91fd None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:58.155555 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-c5104074-0db2-454d-a5b6-6070ca6c91fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:02:58.155555 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:58.155555 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:58.155555 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:58.155555 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:58.155555 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:58.155555 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:58.155555 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:58.155555 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:58.155555 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:58.155555 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:58.155555 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:58.155555 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:02:58.155555 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:02:58.155555 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:02:58.156470 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:02:58.156470 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:58.156470 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:58.156470 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:58.156470 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:58.156470 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:58.156470 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:58.156470 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:58.156470 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:58.156470 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:58.156470 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:58.156470 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:58.156470 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:58.156470 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:58.156470 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:58.156470 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:58.156470 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:58.157357 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:02:58.157357 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:02:58.157357 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:02:58.157357 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:02:58.157357 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:02:58.157357 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:02:58.157357 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:02:58.157357 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:58.160413 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 340/696] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:02:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:58.162247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2a7b0-99ec-464d-8116-b1b0e0bd4609 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:58.162838 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2a7b0-99ec-464d-8116-b1b0e0bd4609 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:58.163381 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2a7b0-99ec-464d-8116-b1b0e0bd4609 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:58.169522 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 341/697] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:58.169522 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e2a7b0-99ec-464d-8116-b1b0e0bd4609 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:58.201215 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19393ed0-0777-4905-bfb2-a5619734061d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:58.203437 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19393ed0-0777-4905-bfb2-a5619734061d None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:58.294010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:58.294730 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e2a7b0-99ec-464d-8116-b1b0e0bd4609 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:58.307774 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:58.334910 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82214d-fd03-4a10-81bd-d00bc6e71671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:58.336069 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82214d-fd03-4a10-81bd-d00bc6e71671 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:58.336908 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82214d-fd03-4a10-81bd-d00bc6e71671 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:58.337644 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82214d-fd03-4a10-81bd-d00bc6e71671 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:58.338638 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c82214d-fd03-4a10-81bd-d00bc6e71671 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:58.339172 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c82214d-fd03-4a10-81bd-d00bc6e71671 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:58.340939 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c82214d-fd03-4a10-81bd-d00bc6e71671 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:58.343365 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 357/698] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:02:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 03 16:02:58.344914 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b3804-264c-42f1-81eb-6012514cbed9 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:58.346220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b3804-264c-42f1-81eb-6012514cbed9 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:58.347698 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b3804-264c-42f1-81eb-6012514cbed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:58.363542 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 342/699] 10.222.0.41 () {56 vars in 1148 bytes} [Fri Jun 3 16:02:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:58.363542 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82214d-fd03-4a10-81bd-d00bc6e71671 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:58.372952 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-275b3804-264c-42f1-81eb-6012514cbed9 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:58.374250 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-275b3804-264c-42f1-81eb-6012514cbed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:58.459332 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 358/700] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:58.460554 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b3804-264c-42f1-81eb-6012514cbed9 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:58.501679 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82214d-fd03-4a10-81bd-d00bc6e71671 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:58.547400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133830d-3df3-4109-92ac-7c70121b7c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:58.548210 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133830d-3df3-4109-92ac-7c70121b7c5b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:58.548533 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133830d-3df3-4109-92ac-7c70121b7c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:58.548758 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133830d-3df3-4109-92ac-7c70121b7c5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:58.561141 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c133830d-3df3-4109-92ac-7c70121b7c5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:58.562691 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c133830d-3df3-4109-92ac-7c70121b7c5b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:58.565207 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:58.572087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b3804-264c-42f1-81eb-6012514cbed9 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:58.584150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-c133830d-3df3-4109-92ac-7c70121b7c5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:58.584150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:58.584150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:58.584150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:58.584150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:58.584150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:58.584150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:58.584150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:58.584150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:58.584150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:58.584150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:58.584150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:58.584150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:02:58.584150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:02:58.584150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:02:58.585129 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:02:58.585129 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:58.585129 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:58.585129 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:58.585129 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:58.585129 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:58.585129 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:58.585129 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:58.585129 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:58.585129 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:58.585129 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:58.585129 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:58.585129 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:58.585129 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:58.585129 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:58.585129 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:58.585129 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:58.586037 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:02:58.586037 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:02:58.586037 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:02:58.586037 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:58.586037 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 343/701] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:58.586864 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133830d-3df3-4109-92ac-7c70121b7c5b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:58.609696 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:58.616748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f3816-0045-4888-ba94-c21cd513e08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:58.617829 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f3816-0045-4888-ba94-c21cd513e08e None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:58.618354 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f3816-0045-4888-ba94-c21cd513e08e None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:58.618835 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f3816-0045-4888-ba94-c21cd513e08e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:58.620026 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f3816-0045-4888-ba94-c21cd513e08e None admin] RBAC: Adding query filter params (role.id=fe695aedbe8249a28c992a19264bc34b, scope.project.id=c274254e510a468e9acbecc6298301cc, user.id=6c5f513b781a4a73b443cded13359eda) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:58.620554 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f3816-0045-4888-ba94-c21cd513e08e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:58.623127 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f3816-0045-4888-ba94-c21cd513e08e None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:58.635427 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 359/702] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:02:58 2022] GET /identity/v3/role_assignments?user.id=6c5f513b781a4a73b443cded13359eda&scope.project.id=c274254e510a468e9acbecc6298301cc&role.id=fe695aedbe8249a28c992a19264bc34b => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:58.636744 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f3816-0045-4888-ba94-c21cd513e08e None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:58.720880 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133830d-3df3-4109-92ac-7c70121b7c5b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:58.763163 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f3816-0045-4888-ba94-c21cd513e08e None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:58.797467 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104231ba-8810-430a-8ede-504fbb79648c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:58.798300 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104231ba-8810-430a-8ede-504fbb79648c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:58.798586 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104231ba-8810-430a-8ede-504fbb79648c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:58.798865 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104231ba-8810-430a-8ede-504fbb79648c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:58.800170 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104231ba-8810-430a-8ede-504fbb79648c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:58.801891 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104231ba-8810-430a-8ede-504fbb79648c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:58.802241 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503900af-8885-419e-b843-1ebb347e74e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:58.803134 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503900af-8885-419e-b843-1ebb347e74e8 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:58.803649 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503900af-8885-419e-b843-1ebb347e74e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:58.804133 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503900af-8885-419e-b843-1ebb347e74e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:58.805061 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503900af-8885-419e-b843-1ebb347e74e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:58.805966 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503900af-8885-419e-b843-1ebb347e74e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:58.807632 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503900af-8885-419e-b843-1ebb347e74e8 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:58.821338 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 360/703] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:58.823311 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-104231ba-8810-430a-8ede-504fbb79648c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 03 16:02:58.823311 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:58.823311 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 16:02:58.823311 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 16:02:58.823311 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 16:02:58.823311 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 16:02:58.823311 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 16:02:58.823311 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:02:58.823311 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 03 16:02:58.823311 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:58.823311 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 16:02:58.823311 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:58.823311 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:58.823311 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:58.823311 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:58.823311 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:58.824239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:58.824239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:58.824239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:58.824239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:58.824239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:58.824239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:58.824239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:58.824239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 16:02:58.824239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 16:02:58.824239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 16:02:58.824239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 16:02:58.824239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:58.824239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:58.824239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:58.824239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:58.824239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:58.825098 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:58.825098 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:58.825098 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:58.825098 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:58.825098 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:58.825098 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:58.825098 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:58.825098 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:58.825098 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:58.825098 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:58.825098 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:58.825098 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 16:02:58.825098 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:02:58.825098 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 03 16:02:58.825098 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:02:58.827402 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503900af-8885-419e-b843-1ebb347e74e8 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:58.833397 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 344/704] 10.222.0.41 () {56 vars in 1129 bytes} [Fri Jun 3 16:02:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:58.843623 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104231ba-8810-430a-8ede-504fbb79648c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:58.877369 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:58.996903 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:59.007060 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503900af-8885-419e-b843-1ebb347e74e8 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:59.016937 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104231ba-8810-430a-8ede-504fbb79648c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:59.056423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a125eb0d-7d04-4878-a1a0-3ccbd621c44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:59.057759 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125eb0d-7d04-4878-a1a0-3ccbd621c44f None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:59.058501 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125eb0d-7d04-4878-a1a0-3ccbd621c44f None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:59.058788 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125eb0d-7d04-4878-a1a0-3ccbd621c44f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:59.060515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584dcbb1-5162-45db-81fa-460f6252cceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:59.061320 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584dcbb1-5162-45db-81fa-460f6252cceb None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:59.061612 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584dcbb1-5162-45db-81fa-460f6252cceb None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:59.061891 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584dcbb1-5162-45db-81fa-460f6252cceb None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:59.062789 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584dcbb1-5162-45db-81fa-460f6252cceb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:59.063105 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584dcbb1-5162-45db-81fa-460f6252cceb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:59.064536 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584dcbb1-5162-45db-81fa-460f6252cceb None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:59.067840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a125eb0d-7d04-4878-a1a0-3ccbd621c44f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:59.069290 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a125eb0d-7d04-4878-a1a0-3ccbd621c44f None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:59.070555 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 345/705] 10.222.0.41 () {56 vars in 1124 bytes} [Fri Jun 3 16:02:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:59.080516 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584dcbb1-5162-45db-81fa-460f6252cceb None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:59.084488 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-a125eb0d-7d04-4878-a1a0-3ccbd621c44f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:02:59.084488 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:02:59.084488 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:02:59.084488 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:02:59.084488 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:02:59.084488 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:02:59.084488 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:02:59.084488 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:02:59.084488 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:02:59.084488 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:02:59.084488 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:02:59.084488 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:02:59.084488 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:02:59.084488 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:02:59.084488 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:02:59.085404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:02:59.085404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:02:59.085404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:02:59.085404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:02:59.085404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:02:59.085404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:02:59.085404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:02:59.085404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:02:59.085404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:02:59.085404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:02:59.085404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:02:59.085404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:02:59.085404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:02:59.085404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:02:59.085404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:02:59.085404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:02:59.085404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:02:59.086276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:02:59.086276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:02:59.086276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:02:59.086276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:02:59.086276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:02:59.086276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:02:59.086276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:02:59.086276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:02:59.086276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 361/706] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:02:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:02:59.091967 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a125eb0d-7d04-4878-a1a0-3ccbd621c44f None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:59.208405 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:59.208608 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584dcbb1-5162-45db-81fa-460f6252cceb None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:59.252982 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a125eb0d-7d04-4878-a1a0-3ccbd621c44f None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:59.256922 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164103b-89e2-4ade-84a1-178d3a9f149d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:59.257591 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164103b-89e2-4ade-84a1-178d3a9f149d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:59.257881 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164103b-89e2-4ade-84a1-178d3a9f149d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:59.258157 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164103b-89e2-4ade-84a1-178d3a9f149d None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:59.258876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b164103b-89e2-4ade-84a1-178d3a9f149d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:59.259329 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b164103b-89e2-4ade-84a1-178d3a9f149d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:59.260731 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b164103b-89e2-4ade-84a1-178d3a9f149d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:59.268813 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 346/707] 10.222.0.41 () {56 vars in 1124 bytes} [Fri Jun 3 16:02:59 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:59.294751 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff43922-7b27-4d80-b246-1631fb007135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:59.295893 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff43922-7b27-4d80-b246-1631fb007135 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:59.296438 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff43922-7b27-4d80-b246-1631fb007135 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:59.296912 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff43922-7b27-4d80-b246-1631fb007135 None admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:59.298052 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff43922-7b27-4d80-b246-1631fb007135 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:59.298576 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff43922-7b27-4d80-b246-1631fb007135 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:59.306635 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff43922-7b27-4d80-b246-1631fb007135 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:59.334554 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 362/708] 10.222.0.41 () {56 vars in 1148 bytes} [Fri Jun 3 16:02:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:02:59.341255 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164103b-89e2-4ade-84a1-178d3a9f149d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:59.370226 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:59.500893 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164103b-89e2-4ade-84a1-178d3a9f149d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:59.542547 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16cfa66-fbf1-442b-9cba-1ca9de342d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:59.543204 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16cfa66-fbf1-442b-9cba-1ca9de342d2e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:59.543539 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16cfa66-fbf1-442b-9cba-1ca9de342d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:59.543813 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16cfa66-fbf1-442b-9cba-1ca9de342d2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:59.544645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16cfa66-fbf1-442b-9cba-1ca9de342d2e None admin] RBAC: Adding query filter params (role.id=75936a608b8149099ff54e317f3a6740, scope.project.id=c274254e510a468e9acbecc6298301cc, user.id=2fd08ea39ee94b92ac6763608e69f158) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:02:59.544954 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16cfa66-fbf1-442b-9cba-1ca9de342d2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:59.546400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16cfa66-fbf1-442b-9cba-1ca9de342d2e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:02:59.553024 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 347/709] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:02:59 2022] GET /identity/v3/role_assignments?user.id=2fd08ea39ee94b92ac6763608e69f158&scope.project.id=c274254e510a468e9acbecc6298301cc&role.id=75936a608b8149099ff54e317f3a6740 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:02:59.567997 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a236882-a340-4853-8d3d-df368becdca4 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:59.568385 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a236882-a340-4853-8d3d-df368becdca4 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:59.568795 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a236882-a340-4853-8d3d-df368becdca4 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:59.570377 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 363/710] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:59.575860 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edaa282-15a8-4f07-92b4-002d89edea53 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:59.576164 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edaa282-15a8-4f07-92b4-002d89edea53 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:59.576426 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edaa282-15a8-4f07-92b4-002d89edea53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:59.594856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-8edaa282-15a8-4f07-92b4-002d89edea53 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:59.596562 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8edaa282-15a8-4f07-92b4-002d89edea53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:59.677622 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 348/711] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:02:59.711541 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a236882-a340-4853-8d3d-df368becdca4 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:59.719074 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442ce2a-e17e-4e5f-ba4c-a391b84dc8b3 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:59.719411 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442ce2a-e17e-4e5f-ba4c-a391b84dc8b3 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:59.719697 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442ce2a-e17e-4e5f-ba4c-a391b84dc8b3 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:59.721381 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 349/712] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:02:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:02:59.727922 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beea49a-07f1-4fd0-8cd7-931e18eacdef None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:59.728921 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beea49a-07f1-4fd0-8cd7-931e18eacdef None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:59.729221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beea49a-07f1-4fd0-8cd7-931e18eacdef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:59.745013 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:02:59.753674 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-6beea49a-07f1-4fd0-8cd7-931e18eacdef None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:02:59.755228 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6beea49a-07f1-4fd0-8cd7-931e18eacdef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:02:59.879267 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 350/713] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:02:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 16:02:59.907942 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beea49a-07f1-4fd0-8cd7-931e18eacdef None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:02:59.911491 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a236882-a340-4853-8d3d-df368becdca4 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:02:59.980106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81037872-4363-4800-a709-1cbcde02e884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:02:59.981246 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81037872-4363-4800-a709-1cbcde02e884 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:02:59.981710 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81037872-4363-4800-a709-1cbcde02e884 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:02:59.982162 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81037872-4363-4800-a709-1cbcde02e884 None admin] PATH_INFO: `/v3/services/network` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:02:59.983036 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81037872-4363-4800-a709-1cbcde02e884 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:02:59.984650 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81037872-4363-4800-a709-1cbcde02e884 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:00.012655 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-81037872-4363-4800-a709-1cbcde02e884 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 03 16:03:00.012655 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:00.012655 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 16:03:00.012655 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 16:03:00.012655 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 16:03:00.012655 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 16:03:00.012655 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 16:03:00.012655 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:00.012655 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 03 16:03:00.012655 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:00.012655 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 16:03:00.012655 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:00.012655 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:00.012655 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:00.012655 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:00.012655 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:00.012655 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:00.013818 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:00.013818 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:00.013818 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:00.013818 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:00.013818 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:00.013818 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:00.013818 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 16:03:00.013818 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 16:03:00.013818 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 16:03:00.013818 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 16:03:00.013818 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:00.013818 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:00.013818 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:00.013818 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:00.013818 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:00.013818 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:00.014672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:00.014672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:00.014672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:00.014672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:00.014672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:00.014672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:00.014672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:00.014672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:00.014672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:00.014672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:00.014672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 16:03:00.014672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:00.014672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 03 16:03:00.014672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:00.019706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 364/714] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:02:59 2022] GET /identity/v3/services/network => generated 88 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:00.025515 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81037872-4363-4800-a709-1cbcde02e884 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:00.036683 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:00.103622 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beea49a-07f1-4fd0-8cd7-931e18eacdef None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:00.139367 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a70ae-ec5b-4e0d-a6c6-b294db35a522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:00.140313 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a70ae-ec5b-4e0d-a6c6-b294db35a522 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:00.140699 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a70ae-ec5b-4e0d-a6c6-b294db35a522 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:00.141080 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a70ae-ec5b-4e0d-a6c6-b294db35a522 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:00.143947 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:00.152824 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a70ae-ec5b-4e0d-a6c6-b294db35a522 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:00.154671 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a70ae-ec5b-4e0d-a6c6-b294db35a522 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:00.157453 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81037872-4363-4800-a709-1cbcde02e884 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:00.173455 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-b42a70ae-ec5b-4e0d-a6c6-b294db35a522 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:00.173455 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:00.173455 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:00.173455 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:00.173455 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:00.173455 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:00.173455 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:00.173455 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:00.173455 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:00.173455 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:00.173455 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:00.173455 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:00.173455 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:03:00.173455 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:03:00.173455 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:03:00.174686 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:03:00.174686 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:00.174686 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:00.174686 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:00.174686 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:00.174686 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:00.174686 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:00.174686 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:00.174686 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:00.174686 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:00.174686 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:00.174686 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:00.174686 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:00.174686 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:00.174686 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:00.174686 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:00.174686 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:00.176323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:03:00.176323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:03:00.176323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:00.176323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:00.183317 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 351/715] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:02:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:03:00.183649 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a70ae-ec5b-4e0d-a6c6-b294db35a522 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:00.195121 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06afe86c-cb05-4c38-896a-bc910fd5edcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:00.195835 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afe86c-cb05-4c38-896a-bc910fd5edcd None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:00.196125 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afe86c-cb05-4c38-896a-bc910fd5edcd None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:00.196766 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afe86c-cb05-4c38-896a-bc910fd5edcd None admin] PATH_INFO: `/v3/services` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:00.197831 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06afe86c-cb05-4c38-896a-bc910fd5edcd None admin] RBAC: Adding query filter params (name=network) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:00.198150 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06afe86c-cb05-4c38-896a-bc910fd5edcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:00.199732 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06afe86c-cb05-4c38-896a-bc910fd5edcd None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:00.206521 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 365/716] 10.222.0.41 () {56 vars in 1112 bytes} [Fri Jun 3 16:03:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:00.213566 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06afe86c-cb05-4c38-896a-bc910fd5edcd None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:00.301740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:00.342275 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06afe86c-cb05-4c38-896a-bc910fd5edcd None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:00.382279 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87d2ef-6571-4356-9abc-68f45a521016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:00.382973 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87d2ef-6571-4356-9abc-68f45a521016 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:00.383260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87d2ef-6571-4356-9abc-68f45a521016 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:00.383586 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87d2ef-6571-4356-9abc-68f45a521016 None admin] PATH_INFO: `/v3/services` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:00.385413 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea87d2ef-6571-4356-9abc-68f45a521016 None admin] RBAC: Adding query filter params (type=network) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:00.385744 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea87d2ef-6571-4356-9abc-68f45a521016 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:00.387149 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea87d2ef-6571-4356-9abc-68f45a521016 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:00.394463 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 366/717] 10.222.0.41 () {56 vars in 1112 bytes} [Fri Jun 3 16:03:00 2022] GET /identity/v3/services?type=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:00.451590 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a70ae-ec5b-4e0d-a6c6-b294db35a522 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:00.511667 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4ae2d-1e59-4540-8f82-8f92728bcca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:00.512352 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4ae2d-1e59-4540-8f82-8f92728bcca0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:00.512636 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4ae2d-1e59-4540-8f82-8f92728bcca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:00.512912 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4ae2d-1e59-4540-8f82-8f92728bcca0 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:00.513666 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e4ae2d-1e59-4540-8f82-8f92728bcca0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:00.513977 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e4ae2d-1e59-4540-8f82-8f92728bcca0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:00.515420 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e4ae2d-1e59-4540-8f82-8f92728bcca0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:00.529757 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 352/718] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:03:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:00.535850 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87d2ef-6571-4356-9abc-68f45a521016 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:00.657630 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87d2ef-6571-4356-9abc-68f45a521016 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:00.690084 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:00.697071 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc9d48c-3c25-48ba-8aed-b321a417dd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:00.697976 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9d48c-3c25-48ba-8aed-b321a417dd66 None admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:00.698496 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9d48c-3c25-48ba-8aed-b321a417dd66 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:00.698950 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9d48c-3c25-48ba-8aed-b321a417dd66 None admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:00.699959 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc9d48c-3c25-48ba-8aed-b321a417dd66 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:00.701601 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc9d48c-3c25-48ba-8aed-b321a417dd66 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:00.721980 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 367/719] 10.222.0.41 () {60 vars in 1132 bytes} [Fri Jun 3 16:03:00 2022] POST /identity/v3/users => generated 254 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:03:01.699460 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac2b41-55c9-4331-b37a-4d5227b681d0 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:01.701470 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac2b41-55c9-4331-b37a-4d5227b681d0 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:01.701470 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac2b41-55c9-4331-b37a-4d5227b681d0 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:01.702192 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 353/720] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:01.708286 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1decc543-1820-45df-9bcb-efa3061f2a32 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:01.708756 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1decc543-1820-45df-9bcb-efa3061f2a32 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:01.708841 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1decc543-1820-45df-9bcb-efa3061f2a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:01.732270 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-1decc543-1820-45df-9bcb-efa3061f2a32 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:01.733815 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-1decc543-1820-45df-9bcb-efa3061f2a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:01.819900 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 368/721] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:03:01.860814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac2b41-55c9-4331-b37a-4d5227b681d0 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:01.941658 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:02.016260 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac2b41-55c9-4331-b37a-4d5227b681d0 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:02.031268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358e385-6c8b-4b24-b438-4936c67aace5 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:02.031918 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358e385-6c8b-4b24-b438-4936c67aace5 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:02.032423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358e385-6c8b-4b24-b438-4936c67aace5 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:02.034801 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 369/722] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:02.040176 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e90d03-74ab-420b-a28e-85637355d9a1 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:02.040766 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e90d03-74ab-420b-a28e-85637355d9a1 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:02.041287 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e90d03-74ab-420b-a28e-85637355d9a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:02.059643 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60dd3a-aae5-4354-bb7c-3975849e7f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:02.060788 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60dd3a-aae5-4354-bb7c-3975849e7f6f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:02.060911 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60dd3a-aae5-4354-bb7c-3975849e7f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:02.061104 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60dd3a-aae5-4354-bb7c-3975849e7f6f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:02.079982 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b60dd3a-aae5-4354-bb7c-3975849e7f6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:03:02.082737 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-c4e90d03-74ab-420b-a28e-85637355d9a1 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:02.084087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-c4e90d03-74ab-420b-a28e-85637355d9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:02.088525 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-2b60dd3a-aae5-4354-bb7c-3975849e7f6f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:03:02.088525 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:02.088525 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:02.088525 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:02.088525 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:02.088525 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:02.088525 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:02.088525 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:02.088525 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:02.088525 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:02.088525 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:02.088525 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:02.088525 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:03:02.088525 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:03:02.088525 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:03:02.088525 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:03:02.089502 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:03:02.089502 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:03:02.089502 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:03:02.089502 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:03:02.089502 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:02.089502 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:02.089502 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:02.089502 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:02.089502 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:02.089502 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:02.089502 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:02.089502 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:02.089502 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:02.089502 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:02.089502 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:02.089502 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:02.090411 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:02.090411 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:02.090411 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:02.090411 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:02.090411 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:03:02.090411 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:03:02.090411 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:03:02.090411 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:03:02.090411 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:03:02.090411 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:03:02.090411 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:03:02.090411 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:02.093525 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 354/723] 10.222.0.41 () {56 vars in 1096 bytes} [Fri Jun 3 16:03:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:02.105358 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60dd3a-aae5-4354-bb7c-3975849e7f6f None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:02.205488 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 370/724] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 16:03:02.252947 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e90d03-74ab-420b-a28e-85637355d9a1 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:02.312738 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60dd3a-aae5-4354-bb7c-3975849e7f6f None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:02.323068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:02.351695 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:02.364405 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ae3a27-0b30-4103-9b6d-14c129f78963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:02.365091 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae3a27-0b30-4103-9b6d-14c129f78963 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:02.365377 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae3a27-0b30-4103-9b6d-14c129f78963 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:02.365657 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae3a27-0b30-4103-9b6d-14c129f78963 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:02.366431 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ae3a27-0b30-4103-9b6d-14c129f78963 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:02.366745 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ae3a27-0b30-4103-9b6d-14c129f78963 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:02.368223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ae3a27-0b30-4103-9b6d-14c129f78963 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:02.392965 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 355/725] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:03:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:02.396281 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ae3a27-0b30-4103-9b6d-14c129f78963 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:02.453946 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e90d03-74ab-420b-a28e-85637355d9a1 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:02.529104 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590bf93c-d9e2-4ec3-8e42-8eca30bc5c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:02.530231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590bf93c-d9e2-4ec3-8e42-8eca30bc5c09 None admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:02.530732 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590bf93c-d9e2-4ec3-8e42-8eca30bc5c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:02.531221 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590bf93c-d9e2-4ec3-8e42-8eca30bc5c09 None admin] PATH_INFO: `/v3/services` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:02.532190 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590bf93c-d9e2-4ec3-8e42-8eca30bc5c09 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:02.535192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590bf93c-d9e2-4ec3-8e42-8eca30bc5c09 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:02.563347 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 371/726] 10.222.0.41 () {60 vars in 1141 bytes} [Fri Jun 3 16:03:02 2022] POST /identity/v3/services => generated 251 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:03:02.594969 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ae3a27-0b30-4103-9b6d-14c129f78963 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:02.630485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c0073-75b3-4dbc-8475-00cf6ca6929c None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:02.631187 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c0073-75b3-4dbc-8475-00cf6ca6929c None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:02.631982 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c0073-75b3-4dbc-8475-00cf6ca6929c None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:02.636294 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 372/727] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:02.642299 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236efea-34c4-443b-b85f-fe6538b4ce66 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:02.642635 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236efea-34c4-443b-b85f-fe6538b4ce66 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:02.642876 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236efea-34c4-443b-b85f-fe6538b4ce66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:02.664179 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c528c-f897-41fc-9741-ccfb2ebea157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:02.665000 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c528c-f897-41fc-9741-ccfb2ebea157 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:02.665282 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c528c-f897-41fc-9741-ccfb2ebea157 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:02.665558 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c528c-f897-41fc-9741-ccfb2ebea157 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:02.681497 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-3236efea-34c4-443b-b85f-fe6538b4ce66 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:02.683817 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-3236efea-34c4-443b-b85f-fe6538b4ce66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:02.687727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741c528c-f897-41fc-9741-ccfb2ebea157 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:02.689220 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741c528c-f897-41fc-9741-ccfb2ebea157 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:02.723816 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-741c528c-f897-41fc-9741-ccfb2ebea157 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:02.723816 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:02.723816 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:02.723816 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:02.723816 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:02.723816 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:02.723816 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:02.723816 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:02.723816 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:02.723816 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:02.723816 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:02.723816 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:02.723816 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:03:02.723816 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:03:02.723816 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:03:02.725221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:03:02.725221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:02.725221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:02.725221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:02.725221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:02.725221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:02.725221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:02.725221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:02.725221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:02.725221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:02.725221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:02.725221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:02.725221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:02.725221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:02.725221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:02.725221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:02.725221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:02.726518 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:03:02.726518 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:03:02.726518 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:02.726518 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:02.726518 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 356/728] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:02.733113 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c528c-f897-41fc-9741-ccfb2ebea157 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:02.788121 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:02.788863 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 373/729] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:02 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:03:02.812441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236efea-34c4-443b-b85f-fe6538b4ce66 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:02.866558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:02.897530 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c528c-f897-41fc-9741-ccfb2ebea157 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:02.937084 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c5f05-9d52-4996-8dd1-4f6fec65709f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:02.938106 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c5f05-9d52-4996-8dd1-4f6fec65709f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:02.938637 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c5f05-9d52-4996-8dd1-4f6fec65709f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:02.939105 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c5f05-9d52-4996-8dd1-4f6fec65709f None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:02.940580 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635c5f05-9d52-4996-8dd1-4f6fec65709f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:02.941091 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635c5f05-9d52-4996-8dd1-4f6fec65709f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:02.941255 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236efea-34c4-443b-b85f-fe6538b4ce66 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:02.942834 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635c5f05-9d52-4996-8dd1-4f6fec65709f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:02.961547 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 357/730] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:03:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:02.963075 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c5f05-9d52-4996-8dd1-4f6fec65709f None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:02.984568 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d91e92-c73d-4038-96b9-dd5dbb7ef69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:02.985379 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d91e92-c73d-4038-96b9-dd5dbb7ef69d None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:02.985665 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d91e92-c73d-4038-96b9-dd5dbb7ef69d None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:02.985960 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d91e92-c73d-4038-96b9-dd5dbb7ef69d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:03.000050 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d91e92-c73d-4038-96b9-dd5dbb7ef69d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:03:03.011175 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-17d91e92-c73d-4038-96b9-dd5dbb7ef69d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:03:03.011175 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:03.011175 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:03.011175 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:03.011175 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:03.011175 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:03.011175 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:03.011175 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:03.011175 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:03.011175 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:03.011175 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:03.011175 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:03.011175 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:03:03.011175 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:03:03.011175 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:03:03.011175 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:03:03.012192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:03:03.012192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:03:03.012192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:03:03.012192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:03:03.012192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:03.012192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:03.012192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:03.012192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:03.012192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:03.012192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:03.012192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:03.012192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:03.012192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:03.012192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:03.012192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:03.012192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:03.013231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:03.013231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:03.013231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:03.013231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:03.013231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:03:03.013231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:03:03.013231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:03:03.013231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:03:03.013231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:03:03.013231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:03:03.013231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:03:03.013231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:03.013231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 374/731] 10.222.0.41 () {56 vars in 1096 bytes} [Fri Jun 3 16:03:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:03:03.020056 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d91e92-c73d-4038-96b9-dd5dbb7ef69d None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:03.064767 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:03.096823 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c5f05-9d52-4996-8dd1-4f6fec65709f None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:03.135522 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:03.139509 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c91c6a-df80-4cd1-a9ed-d15c0254885b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:03.140309 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c91c6a-df80-4cd1-a9ed-d15c0254885b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:03.140596 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c91c6a-df80-4cd1-a9ed-d15c0254885b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:03.140876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c91c6a-df80-4cd1-a9ed-d15c0254885b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:03.141233 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d91e92-c73d-4038-96b9-dd5dbb7ef69d None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:03.149161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c91c6a-df80-4cd1-a9ed-d15c0254885b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:03.150857 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c91c6a-df80-4cd1-a9ed-d15c0254885b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:03.169289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-53c91c6a-df80-4cd1-a9ed-d15c0254885b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 03 16:03:03.169289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:03.169289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:03.169289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:03.169289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:03.169289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:03.169289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:03.169289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:03.169289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:03.169289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:03.169289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:03.169289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:03.169289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:03:03.169289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:03:03.169289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:03:03.169289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:03:03.170302 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:03.170302 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:03.170302 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:03:03.170302 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:03:03.170302 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:03:03.170302 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:03:03.170302 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:03.170302 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:03.170302 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:03.170302 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:03.170302 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:03.170302 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:03.170302 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:03.170302 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:03.170302 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:03.170302 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:03.170302 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:03.170302 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:03.171459 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:03.171459 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:03.171459 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:03:03.171459 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:03:03.171459 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:03:03.171459 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:03:03.171459 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:03:03.171459 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:03:03.171459 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 03 16:03:03.171459 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:03.171459 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 358/732] 10.222.0.41 () {56 vars in 1096 bytes} [Fri Jun 3 16:03:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:03:03.176824 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef3609-4995-4d97-ac10-5cf57096b5ee None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:03.176824 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef3609-4995-4d97-ac10-5cf57096b5ee None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:03.176824 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef3609-4995-4d97-ac10-5cf57096b5ee None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:03.176824 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 359/733] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:03.180272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ef3609-4995-4d97-ac10-5cf57096b5ee None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:03.182770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988793c8-72d9-4d9b-8a88-31eaec549605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:03.183598 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988793c8-72d9-4d9b-8a88-31eaec549605 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:03.183889 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988793c8-72d9-4d9b-8a88-31eaec549605 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:03.184168 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988793c8-72d9-4d9b-8a88-31eaec549605 None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:03.185038 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988793c8-72d9-4d9b-8a88-31eaec549605 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:03.185352 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988793c8-72d9-4d9b-8a88-31eaec549605 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:03.186755 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988793c8-72d9-4d9b-8a88-31eaec549605 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:03.196394 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 375/734] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:03:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:03.197662 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e679451-790e-4810-814d-c68dadcb6a03 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:03.197956 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e679451-790e-4810-814d-c68dadcb6a03 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:03.198233 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e679451-790e-4810-814d-c68dadcb6a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:03.228326 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-0e679451-790e-4810-814d-c68dadcb6a03 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:03.229774 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-0e679451-790e-4810-814d-c68dadcb6a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:03.304789 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ef3609-4995-4d97-ac10-5cf57096b5ee None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:03.313063 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 376/735] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:03 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 16:03:03.314377 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e679451-790e-4810-814d-c68dadcb6a03 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:03.342140 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb7a7e-017c-4786-8a70-977ee6d716e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:03.342813 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb7a7e-017c-4786-8a70-977ee6d716e5 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:03.343096 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb7a7e-017c-4786-8a70-977ee6d716e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:03.343566 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb7a7e-017c-4786-8a70-977ee6d716e5 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:03.344733 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb7a7e-017c-4786-8a70-977ee6d716e5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:03.345053 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb7a7e-017c-4786-8a70-977ee6d716e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:03.346834 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb7a7e-017c-4786-8a70-977ee6d716e5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:03.362959 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:03.379861 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 360/736] 10.222.0.41 () {56 vars in 1135 bytes} [Fri Jun 3 16:03:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:03.379861 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb7a7e-017c-4786-8a70-977ee6d716e5 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:03.433934 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:03.447466 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e679451-790e-4810-814d-c68dadcb6a03 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:03.483633 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8750d0-21ff-4bb6-ab0b-73e5eeb7f777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:03.484570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8750d0-21ff-4bb6-ab0b-73e5eeb7f777 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:03.485049 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8750d0-21ff-4bb6-ab0b-73e5eeb7f777 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:03.485328 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8750d0-21ff-4bb6-ab0b-73e5eeb7f777 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:03.492253 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb7a7e-017c-4786-8a70-977ee6d716e5 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:03.497064 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8750d0-21ff-4bb6-ab0b-73e5eeb7f777 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:03.498622 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8750d0-21ff-4bb6-ab0b-73e5eeb7f777 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:03.517315 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-3a8750d0-21ff-4bb6-ab0b-73e5eeb7f777 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:03.517315 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:03.517315 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:03.517315 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:03.517315 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:03.517315 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:03.517315 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:03.517315 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:03.517315 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:03.517315 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:03.517315 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:03.517315 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:03.517315 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:03:03.517315 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:03:03.517315 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:03:03.519022 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:03:03.519022 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:03.519022 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:03.519022 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:03.519022 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:03.519022 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:03.519022 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:03.519022 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:03.519022 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:03.519022 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:03.519022 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:03.519022 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:03.519022 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:03.519022 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:03.519022 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:03.519022 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:03.519022 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:03.520130 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:03:03.520130 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:03:03.520130 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:03.520130 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:03.525908 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 377/737] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:03.525908 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8750d0-21ff-4bb6-ab0b-73e5eeb7f777 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:03.542225 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b8e632-5359-44fc-8517-514c21bb93de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:03.543322 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b8e632-5359-44fc-8517-514c21bb93de None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:03.544062 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b8e632-5359-44fc-8517-514c21bb93de None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:03.544504 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b8e632-5359-44fc-8517-514c21bb93de None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:03.545387 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b8e632-5359-44fc-8517-514c21bb93de None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:03.547338 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b8e632-5359-44fc-8517-514c21bb93de None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:03.565828 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 361/738] 10.222.0.41 () {60 vars in 1141 bytes} [Fri Jun 3 16:03:03 2022] POST /identity/v3/services => generated 240 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:03:03.566902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b8e632-5359-44fc-8517-514c21bb93de None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:03.643142 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:03.663271 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8750d0-21ff-4bb6-ab0b-73e5eeb7f777 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:03.714008 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b8e632-5359-44fc-8517-514c21bb93de None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:03.718216 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d2a46-a14b-47e1-b696-aa96d3aa167c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:03.719134 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d2a46-a14b-47e1-b696-aa96d3aa167c None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:03.719592 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d2a46-a14b-47e1-b696-aa96d3aa167c None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:03.720034 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d2a46-a14b-47e1-b696-aa96d3aa167c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:03.731801 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:03.764570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a37b3e-b00d-4ca6-b6ac-5628146bd4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:03.765274 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a37b3e-b00d-4ca6-b6ac-5628146bd4ff None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:03.765536 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a37b3e-b00d-4ca6-b6ac-5628146bd4ff None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:03.765813 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a37b3e-b00d-4ca6-b6ac-5628146bd4ff None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:03.766630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a37b3e-b00d-4ca6-b6ac-5628146bd4ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:03.766942 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a37b3e-b00d-4ca6-b6ac-5628146bd4ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:03.769747 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a37b3e-b00d-4ca6-b6ac-5628146bd4ff None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:03.787418 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 362/739] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:03:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:03.792802 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b987fb3d-d715-42bf-af8f-18dfaea3ce1e None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:03.794013 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b987fb3d-d715-42bf-af8f-18dfaea3ce1e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:03.795142 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b987fb3d-d715-42bf-af8f-18dfaea3ce1e None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:03.796644 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 363/740] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:03.797703 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b987fb3d-d715-42bf-af8f-18dfaea3ce1e None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:03.812011 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016d2a46-a14b-47e1-b696-aa96d3aa167c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:03.814125 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016d2a46-a14b-47e1-b696-aa96d3aa167c None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:03.924709 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 378/741] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:03:03 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 16:03:03.925593 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b987fb3d-d715-42bf-af8f-18dfaea3ce1e None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:03.926603 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb374df-2302-4762-af74-bb0da7637358 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:03.927094 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb374df-2302-4762-af74-bb0da7637358 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:03.927609 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb374df-2302-4762-af74-bb0da7637358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:03.939997 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:03.945168 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-beb374df-2302-4762-af74-bb0da7637358 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:03.946611 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-beb374df-2302-4762-af74-bb0da7637358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:03.964228 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d8a88-a145-42d2-bc89-5899d5defce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:03.965036 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d8a88-a145-42d2-bc89-5899d5defce6 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:03.965322 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d8a88-a145-42d2-bc89-5899d5defce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:03.965598 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d8a88-a145-42d2-bc89-5899d5defce6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:03.973396 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d8a88-a145-42d2-bc89-5899d5defce6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:03.974843 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d8a88-a145-42d2-bc89-5899d5defce6 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:03.979186 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:03.993173 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-5e5d8a88-a145-42d2-bc89-5899d5defce6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 03 16:03:03.993173 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:03.993173 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:03.993173 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:03.993173 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:03.993173 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:03.993173 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:03.993173 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:03.993173 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:03.993173 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:03.993173 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:03.993173 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:03.993173 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:03:03.993173 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:03:03.993173 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:03:03.993173 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:03:03.994201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:03.994201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:03.994201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:03:03.994201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:03:03.994201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:03:03.994201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:03:03.994201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:03.994201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:03.994201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:03.994201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:03.994201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:03.994201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:03.994201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:03.994201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:03.994201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:03.994201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:03.994201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:03.994201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:03.995141 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:03.995141 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:03.995141 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:03:03.995141 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:03:03.995141 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:03:03.995141 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:03:03.995141 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:03:03.995141 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:03:03.995141 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 03 16:03:03.995141 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:03.995141 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 364/742] 10.222.0.41 () {56 vars in 1093 bytes} [Fri Jun 3 16:03:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:03.995141 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d8a88-a145-42d2-bc89-5899d5defce6 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:04.036484 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 379/743] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:03:04.037844 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb374df-2302-4762-af74-bb0da7637358 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:04.100113 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d8a88-a145-42d2-bc89-5899d5defce6 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:04.135912 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46af621-c32c-4303-b83f-5b4a840c877d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:04.136752 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46af621-c32c-4303-b83f-5b4a840c877d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:04.137033 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46af621-c32c-4303-b83f-5b4a840c877d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:04.137305 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46af621-c32c-4303-b83f-5b4a840c877d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:04.147723 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46af621-c32c-4303-b83f-5b4a840c877d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:04.149099 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46af621-c32c-4303-b83f-5b4a840c877d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:04.151777 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb374df-2302-4762-af74-bb0da7637358 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:04.168002 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-c46af621-c32c-4303-b83f-5b4a840c877d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:04.168002 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:04.168002 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:04.168002 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:04.168002 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:04.168002 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:04.168002 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:04.168002 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:04.168002 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:04.168002 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:04.168002 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:04.168002 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:04.168002 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:03:04.168002 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:03:04.168002 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:03:04.168956 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:03:04.168956 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:04.168956 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:04.168956 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:04.168956 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:04.168956 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:04.168956 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:04.168956 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:04.168956 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:04.168956 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:04.168956 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:04.168956 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:04.168956 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:04.168956 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:04.168956 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:04.168956 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:04.168956 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:04.169849 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:03:04.169849 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:03:04.169849 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:04.169849 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:04.169849 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 365/744] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:04.170547 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46af621-c32c-4303-b83f-5b4a840c877d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:04.195227 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab769871-2b84-44b5-b8f5-1645309d7dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:04.196345 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab769871-2b84-44b5-b8f5-1645309d7dce None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:04.196844 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab769871-2b84-44b5-b8f5-1645309d7dce None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:04.197345 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab769871-2b84-44b5-b8f5-1645309d7dce None admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:04.198399 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab769871-2b84-44b5-b8f5-1645309d7dce None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:04.198959 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab769871-2b84-44b5-b8f5-1645309d7dce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:04.200710 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab769871-2b84-44b5-b8f5-1645309d7dce None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:04.205623 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:04.220887 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 380/745] 10.222.0.41 () {56 vars in 1133 bytes} [Fri Jun 3 16:03:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:04.222184 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab769871-2b84-44b5-b8f5-1645309d7dce None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:04.252111 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:04.309432 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46af621-c32c-4303-b83f-5b4a840c877d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:04.330220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab769871-2b84-44b5-b8f5-1645309d7dce None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:04.347274 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2033dbc1-8067-406a-bfb2-bbb1c05286a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:04.348407 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2033dbc1-8067-406a-bfb2-bbb1c05286a2 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:04.348955 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2033dbc1-8067-406a-bfb2-bbb1c05286a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:04.349545 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2033dbc1-8067-406a-bfb2-bbb1c05286a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:04.360819 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2033dbc1-8067-406a-bfb2-bbb1c05286a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:03:04.367305 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a33d15-5f15-44ee-b213-8f9e582efad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:04.368053 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a33d15-5f15-44ee-b213-8f9e582efad6 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:04.368412 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a33d15-5f15-44ee-b213-8f9e582efad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:04.368762 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a33d15-5f15-44ee-b213-8f9e582efad6 None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:04.369554 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a33d15-5f15-44ee-b213-8f9e582efad6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:04.369954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a33d15-5f15-44ee-b213-8f9e582efad6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:04.370856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-2033dbc1-8067-406a-bfb2-bbb1c05286a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:03:04.370856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:04.370856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:04.370856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:04.370856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:04.370856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:04.370856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:04.370856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:04.370856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:04.370856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:04.370856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:04.370856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:04.370856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:03:04.370856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:03:04.370856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:03:04.370856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:03:04.371793 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:03:04.371793 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:03:04.371793 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:03:04.371793 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:03:04.371793 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:04.371793 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:04.371793 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:04.371793 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:04.371793 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:04.371793 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:04.371793 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:04.371793 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:04.371793 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:04.371793 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:04.371793 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:04.371793 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:04.372645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:04.372645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:04.372645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:04.372645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:04.372645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:03:04.372645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:03:04.372645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:03:04.372645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:03:04.372645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:03:04.372645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:03:04.372645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:03:04.372645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:04.373383 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a33d15-5f15-44ee-b213-8f9e582efad6 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:04.375729 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 366/746] 10.222.0.41 () {56 vars in 1102 bytes} [Fri Jun 3 16:03:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:04.376803 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2033dbc1-8067-406a-bfb2-bbb1c05286a2 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:04.389111 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 381/747] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:03:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:04.389111 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a33d15-5f15-44ee-b213-8f9e582efad6 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:04.475449 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:04.476689 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:04.494602 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a33d15-5f15-44ee-b213-8f9e582efad6 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:04.496443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2033dbc1-8067-406a-bfb2-bbb1c05286a2 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:04.531042 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6471b-d236-4f0c-9ec8-0a3a0db07ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:04.531862 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6471b-d236-4f0c-9ec8-0a3a0db07ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:04.532217 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6471b-d236-4f0c-9ec8-0a3a0db07ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:04.532567 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6471b-d236-4f0c-9ec8-0a3a0db07ac3 None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:04.533388 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b6471b-d236-4f0c-9ec8-0a3a0db07ac3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:04.534281 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b6471b-d236-4f0c-9ec8-0a3a0db07ac3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:04.534800 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6403ebfe-7f1b-47fe-9efa-8715e38d682a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:04.535811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b6471b-d236-4f0c-9ec8-0a3a0db07ac3 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:04.535997 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403ebfe-7f1b-47fe-9efa-8715e38d682a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:04.536470 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403ebfe-7f1b-47fe-9efa-8715e38d682a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:04.536921 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403ebfe-7f1b-47fe-9efa-8715e38d682a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:04.545376 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 382/748] 10.222.0.41 () {56 vars in 1103 bytes} [Fri Jun 3 16:03:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:04.546210 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6471b-d236-4f0c-9ec8-0a3a0db07ac3 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:04.550609 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6403ebfe-7f1b-47fe-9efa-8715e38d682a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:04.552449 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6403ebfe-7f1b-47fe-9efa-8715e38d682a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:04.572341 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-6403ebfe-7f1b-47fe-9efa-8715e38d682a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:04.572341 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:04.572341 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:04.572341 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:04.572341 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:04.572341 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:04.572341 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:04.572341 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:04.572341 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:04.572341 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:04.572341 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:04.572341 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:04.572341 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:03:04.572341 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:03:04.572341 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:03:04.573264 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:03:04.573264 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:04.573264 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:04.573264 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:04.573264 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:04.573264 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:04.573264 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:04.573264 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:04.573264 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:04.573264 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:04.573264 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:04.573264 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:04.573264 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:04.573264 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:04.573264 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:04.573264 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:04.573264 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:04.574151 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:03:04.574151 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:03:04.574151 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:04.574151 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:04.580969 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 367/749] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:04.582125 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6403ebfe-7f1b-47fe-9efa-8715e38d682a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:04.652380 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6471b-d236-4f0c-9ec8-0a3a0db07ac3 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:04.689520 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65fd421-ca76-427d-879a-47f167f8e4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:04.690425 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fd421-ca76-427d-879a-47f167f8e4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:04.690717 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fd421-ca76-427d-879a-47f167f8e4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:04.690998 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fd421-ca76-427d-879a-47f167f8e4b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:04.692562 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6403ebfe-7f1b-47fe-9efa-8715e38d682a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:04.698118 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65fd421-ca76-427d-879a-47f167f8e4b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:04.699583 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65fd421-ca76-427d-879a-47f167f8e4b3 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:04.715526 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-a65fd421-ca76-427d-879a-47f167f8e4b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:03:04.715526 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:04.715526 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:04.715526 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:04.715526 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:04.715526 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:04.715526 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:04.715526 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:04.715526 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:04.715526 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:04.715526 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:04.715526 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:04.715526 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:03:04.715526 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:03:04.715526 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:03:04.716507 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:03:04.716507 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:04.716507 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:04.716507 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:04.716507 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:04.716507 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:04.716507 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:04.716507 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:04.716507 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:04.716507 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:04.716507 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:04.716507 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:04.716507 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:04.716507 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:04.716507 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:04.716507 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:04.716507 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:04.717840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:03:04.717840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:03:04.717840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:03:04.717840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:03:04.717840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:03:04.717840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:03:04.717840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:03:04.717840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:04.717840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 383/750] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:03:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:04.717840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65fd421-ca76-427d-879a-47f167f8e4b3 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:04.732884 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420aa6b-ad13-4406-a801-27d33af0848c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:04.734128 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:04.734964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420aa6b-ad13-4406-a801-27d33af0848c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:04.735495 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420aa6b-ad13-4406-a801-27d33af0848c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:04.736045 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420aa6b-ad13-4406-a801-27d33af0848c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:04.740881 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:04.750684 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8420aa6b-ad13-4406-a801-27d33af0848c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:04.752831 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8420aa6b-ad13-4406-a801-27d33af0848c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:04.772270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-8420aa6b-ad13-4406-a801-27d33af0848c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:04.772270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:04.772270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:04.772270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:04.772270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:04.772270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:04.772270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:04.772270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:04.772270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:04.772270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:04.772270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:04.772270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:04.772270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:03:04.772270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:03:04.772270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:03:04.773601 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:03:04.773601 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:04.773601 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:04.773601 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:04.773601 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:04.773601 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:04.773601 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:04.773601 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:04.773601 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:04.773601 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:04.773601 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:04.773601 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:04.773601 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:04.773601 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:04.773601 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:04.773601 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:04.773601 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:04.774476 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:03:04.774476 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:03:04.774476 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:04.774476 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:04.777019 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 368/751] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:04.778189 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420aa6b-ad13-4406-a801-27d33af0848c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:04.833665 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65fd421-ca76-427d-879a-47f167f8e4b3 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:04.871088 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7bce0c-3b72-4dcd-a258-923d133d5a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:04.871800 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7bce0c-3b72-4dcd-a258-923d133d5a33 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:04.872085 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7bce0c-3b72-4dcd-a258-923d133d5a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:04.872362 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7bce0c-3b72-4dcd-a258-923d133d5a33 None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:04.873157 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7bce0c-3b72-4dcd-a258-923d133d5a33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:04.873417 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7bce0c-3b72-4dcd-a258-923d133d5a33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:04.875012 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7bce0c-3b72-4dcd-a258-923d133d5a33 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:04.893439 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 384/752] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:03:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:04.893439 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7bce0c-3b72-4dcd-a258-923d133d5a33 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:04.902734 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420aa6b-ad13-4406-a801-27d33af0848c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:04.940979 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9dbbab-7a3b-4ecc-8071-a5fc6064fe80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:04.942069 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9dbbab-7a3b-4ecc-8071-a5fc6064fe80 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:04.942618 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9dbbab-7a3b-4ecc-8071-a5fc6064fe80 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:04.943099 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9dbbab-7a3b-4ecc-8071-a5fc6064fe80 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:04.944464 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9dbbab-7a3b-4ecc-8071-a5fc6064fe80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:04.945118 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9dbbab-7a3b-4ecc-8071-a5fc6064fe80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:04.946802 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9dbbab-7a3b-4ecc-8071-a5fc6064fe80 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:04.959106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:04.964035 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 369/753] 10.222.0.41 () {56 vars in 1148 bytes} [Fri Jun 3 16:03:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:04.965194 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9dbbab-7a3b-4ecc-8071-a5fc6064fe80 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:04.999184 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7bce0c-3b72-4dcd-a258-923d133d5a33 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:05.027770 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:05.037383 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd5559-19e4-42f0-841e-679285139cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:05.038060 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd5559-19e4-42f0-841e-679285139cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:05.038346 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd5559-19e4-42f0-841e-679285139cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:05.038626 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd5559-19e4-42f0-841e-679285139cb4 None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:05.039376 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd5559-19e4-42f0-841e-679285139cb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:05.039689 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd5559-19e4-42f0-841e-679285139cb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:05.041423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd5559-19e4-42f0-841e-679285139cb4 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:05.052905 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 385/754] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:03:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:05.053800 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd5559-19e4-42f0-841e-679285139cb4 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:05.078074 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9dbbab-7a3b-4ecc-8071-a5fc6064fe80 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:05.116600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11abd89a-6c34-420d-9610-71ed3dcad000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:05.117753 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abd89a-6c34-420d-9610-71ed3dcad000 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:05.118314 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abd89a-6c34-420d-9610-71ed3dcad000 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:05.118845 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abd89a-6c34-420d-9610-71ed3dcad000 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:05.126554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11abd89a-6c34-420d-9610-71ed3dcad000 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:05.128899 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11abd89a-6c34-420d-9610-71ed3dcad000 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:05.145029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-11abd89a-6c34-420d-9610-71ed3dcad000 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:03:05.145029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:05.145029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:05.145029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:05.145029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:05.145029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:05.145029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:05.145029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:05.145029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:05.145029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:05.145029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:05.145029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:05.145029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:03:05.145029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:03:05.145029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:03:05.145948 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:03:05.145948 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:05.145948 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:05.145948 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:05.145948 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:05.145948 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:05.145948 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:05.145948 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:05.145948 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:05.145948 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:05.145948 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:05.145948 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:05.145948 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:05.145948 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:05.145948 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:05.145948 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:05.145948 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:05.146865 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:03:05.146865 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:03:05.146865 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:03:05.146865 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:03:05.146865 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:03:05.146865 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:03:05.146865 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:03:05.146865 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:05.154291 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 370/755] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:03:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:05.155510 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11abd89a-6c34-420d-9610-71ed3dcad000 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:05.156676 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd5559-19e4-42f0-841e-679285139cb4 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:05.184664 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:05.192699 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a41777-177e-40ad-9d4b-6ce763813797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:05.193410 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a41777-177e-40ad-9d4b-6ce763813797 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:05.193637 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a41777-177e-40ad-9d4b-6ce763813797 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:05.193914 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a41777-177e-40ad-9d4b-6ce763813797 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:05.194764 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a41777-177e-40ad-9d4b-6ce763813797 None admin] RBAC: Adding query filter params (role.id=75936a608b8149099ff54e317f3a6740, scope.project.id=c274254e510a468e9acbecc6298301cc, user.id=6c5f513b781a4a73b443cded13359eda) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:05.195072 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a41777-177e-40ad-9d4b-6ce763813797 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:05.196531 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a41777-177e-40ad-9d4b-6ce763813797 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:05.207484 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 386/756] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:03:05 2022] GET /identity/v3/role_assignments?user.id=6c5f513b781a4a73b443cded13359eda&scope.project.id=c274254e510a468e9acbecc6298301cc&role.id=75936a608b8149099ff54e317f3a6740 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:05.208547 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a41777-177e-40ad-9d4b-6ce763813797 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:05.292836 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11abd89a-6c34-420d-9610-71ed3dcad000 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:05.307080 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:05.343536 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fcab4a-3917-42d7-8e16-0aa781effc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:05.344382 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fcab4a-3917-42d7-8e16-0aa781effc17 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:05.344672 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fcab4a-3917-42d7-8e16-0aa781effc17 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:05.345216 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fcab4a-3917-42d7-8e16-0aa781effc17 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:05.353353 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a41777-177e-40ad-9d4b-6ce763813797 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:05.357344 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fcab4a-3917-42d7-8e16-0aa781effc17 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:05.358841 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fcab4a-3917-42d7-8e16-0aa781effc17 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:05.378056 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-b0fcab4a-3917-42d7-8e16-0aa781effc17 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 03 16:03:05.378056 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:05.378056 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:05.378056 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:05.378056 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:05.378056 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:05.378056 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:05.378056 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:05.378056 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:05.378056 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:05.378056 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:05.378056 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:05.378056 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:03:05.378056 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:03:05.378056 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:03:05.378056 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:03:05.379062 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:05.379062 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:05.379062 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:03:05.379062 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:03:05.379062 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:03:05.379062 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:03:05.379062 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:05.379062 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:05.379062 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:05.379062 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:05.379062 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:05.379062 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:05.379062 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:05.379062 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:05.379062 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:05.379062 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:05.379062 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:05.379062 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:05.381039 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:05.381039 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:05.381039 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:03:05.381039 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:03:05.381039 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:03:05.381039 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:03:05.381039 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:03:05.381039 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:03:05.381039 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 03 16:03:05.381039 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:05.381039 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 371/757] 10.222.0.41 () {56 vars in 1117 bytes} [Fri Jun 3 16:03:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:05.382238 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3477443-d8d8-452e-8bee-a70380acba81 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:05.382238 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3477443-d8d8-452e-8bee-a70380acba81 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:05.383779 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3477443-d8d8-452e-8bee-a70380acba81 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:05.387560 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 372/758] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:05.388472 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3477443-d8d8-452e-8bee-a70380acba81 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:05.399497 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de46ebfc-c5b6-472a-9c76-5d4f947298fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:05.400238 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46ebfc-c5b6-472a-9c76-5d4f947298fe None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:05.400524 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46ebfc-c5b6-472a-9c76-5d4f947298fe None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:05.400801 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46ebfc-c5b6-472a-9c76-5d4f947298fe None admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:05.401718 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de46ebfc-c5b6-472a-9c76-5d4f947298fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:05.402034 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de46ebfc-c5b6-472a-9c76-5d4f947298fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:05.403938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de46ebfc-c5b6-472a-9c76-5d4f947298fe None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:05.419268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 387/759] 10.222.0.41 () {56 vars in 1148 bytes} [Fri Jun 3 16:03:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:05.420864 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31351f56-99cc-425f-b077-0e1394bf1ee0 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:05.421153 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31351f56-99cc-425f-b077-0e1394bf1ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:05.421436 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31351f56-99cc-425f-b077-0e1394bf1ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:05.439745 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-31351f56-99cc-425f-b077-0e1394bf1ee0 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:05.441492 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-31351f56-99cc-425f-b077-0e1394bf1ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:05.493951 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:05.515968 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3477443-d8d8-452e-8bee-a70380acba81 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:05.521640 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 388/760] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:03:05.522467 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31351f56-99cc-425f-b077-0e1394bf1ee0 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:05.567973 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8582f739-606d-4a05-a399-6b0bca4494a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:05.568642 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582f739-606d-4a05-a399-6b0bca4494a1 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:05.568987 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582f739-606d-4a05-a399-6b0bca4494a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:05.569269 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582f739-606d-4a05-a399-6b0bca4494a1 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:05.570162 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8582f739-606d-4a05-a399-6b0bca4494a1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:05.570481 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8582f739-606d-4a05-a399-6b0bca4494a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:05.571957 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8582f739-606d-4a05-a399-6b0bca4494a1 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:05.594081 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 373/761] 10.222.0.41 () {56 vars in 1149 bytes} [Fri Jun 3 16:03:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:05.595518 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8582f739-606d-4a05-a399-6b0bca4494a1 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:05.631853 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:05.648751 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31351f56-99cc-425f-b077-0e1394bf1ee0 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:05.689241 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c5a4e-bc6d-411d-87e8-7d296ae8f1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:05.691166 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c5a4e-bc6d-411d-87e8-7d296ae8f1eb None admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:05.691665 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c5a4e-bc6d-411d-87e8-7d296ae8f1eb None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:05.692091 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c5a4e-bc6d-411d-87e8-7d296ae8f1eb None admin] PATH_INFO: `/v3/projects/c274254e510a468e9acbecc6298301cc/users/2fd08ea39ee94b92ac6763608e69f158/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:05.694493 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c5a4e-bc6d-411d-87e8-7d296ae8f1eb None admin] RBAC: Authorizing `identity:create_grant(project_id=c274254e510a468e9acbecc6298301cc, user_id=2fd08ea39ee94b92ac6763608e69f158, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:05.696356 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c5a4e-bc6d-411d-87e8-7d296ae8f1eb None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:05.713332 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 389/762] 10.222.0.41 () {58 vars in 1439 bytes} [Fri Jun 3 16:03:05 2022] PUT /identity/v3/projects/c274254e510a468e9acbecc6298301cc/users/2fd08ea39ee94b92ac6763608e69f158/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:03:05.714872 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c5a4e-bc6d-411d-87e8-7d296ae8f1eb None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:05.724149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8582f739-606d-4a05-a399-6b0bca4494a1 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:05.769259 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:05.795698 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ef879-fed5-49f1-bcb2-5d1e531cd7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:05.796517 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ef879-fed5-49f1-bcb2-5d1e531cd7f4 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:05.796803 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ef879-fed5-49f1-bcb2-5d1e531cd7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:05.797079 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ef879-fed5-49f1-bcb2-5d1e531cd7f4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:05.798096 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ef879-fed5-49f1-bcb2-5d1e531cd7f4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:05.799238 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ef879-fed5-49f1-bcb2-5d1e531cd7f4 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:05.822937 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-6d6ef879-fed5-49f1-bcb2-5d1e531cd7f4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 03 16:03:05.822937 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:05.822937 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 16:03:05.822937 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 16:03:05.822937 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 16:03:05.822937 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 16:03:05.822937 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 16:03:05.822937 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:05.822937 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 03 16:03:05.822937 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:05.822937 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 16:03:05.822937 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:05.822937 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:05.822937 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:05.822937 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:05.822937 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:05.824092 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:05.824092 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:05.824092 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:05.824092 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:05.824092 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:05.824092 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:05.824092 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:05.824092 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 16:03:05.824092 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 16:03:05.824092 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 16:03:05.824092 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 16:03:05.824092 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:05.824092 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:05.824092 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:05.824092 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:05.824092 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:05.824990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:05.824990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:05.824990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:05.824990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:05.824990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:05.824990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:05.824990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:05.824990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:05.824990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:05.824990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:05.824990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:05.824990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 16:03:05.824990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:05.824990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 03 16:03:05.824990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:05.824990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 374/763] 10.222.0.41 () {56 vars in 1129 bytes} [Fri Jun 3 16:03:05 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:05.832592 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ef879-fed5-49f1-bcb2-5d1e531cd7f4 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:05.866405 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c5a4e-bc6d-411d-87e8-7d296ae8f1eb None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:05.907560 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d0221-60c4-4c53-9877-904dda806723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:05.908226 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d0221-60c4-4c53-9877-904dda806723 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:05.908517 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d0221-60c4-4c53-9877-904dda806723 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:05.909345 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d0221-60c4-4c53-9877-904dda806723 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:06.007334 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:06.019902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ef879-fed5-49f1-bcb2-5d1e531cd7f4 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:06.028290 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86d0221-60c4-4c53-9877-904dda806723 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:06.030404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86d0221-60c4-4c53-9877-904dda806723 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:06.058877 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c96ee14-0826-4a4a-a517-e17f8330167e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:06.059687 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96ee14-0826-4a4a-a517-e17f8330167e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:06.059965 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96ee14-0826-4a4a-a517-e17f8330167e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:06.060275 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96ee14-0826-4a4a-a517-e17f8330167e None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:06.061065 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c96ee14-0826-4a4a-a517-e17f8330167e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:06.061371 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c96ee14-0826-4a4a-a517-e17f8330167e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:06.062753 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c96ee14-0826-4a4a-a517-e17f8330167e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:06.071186 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 375/764] 10.222.0.41 () {56 vars in 1124 bytes} [Fri Jun 3 16:03:05 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:06.071186 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c899e4-d801-4724-b4a2-bea5e8f03759 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:06.071186 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c899e4-d801-4724-b4a2-bea5e8f03759 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:06.071186 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c899e4-d801-4724-b4a2-bea5e8f03759 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:06.071740 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 376/765] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:06.073114 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c899e4-d801-4724-b4a2-bea5e8f03759 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:06.081570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:06.157564 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 390/766] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:03:05 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 16:03:06.159021 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec39826-27dc-49c6-955d-ed74a2fe40e3 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:06.159328 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec39826-27dc-49c6-955d-ed74a2fe40e3 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:06.159617 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec39826-27dc-49c6-955d-ed74a2fe40e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:06.181624 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-9ec39826-27dc-49c6-955d-ed74a2fe40e3 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:06.183101 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-9ec39826-27dc-49c6-955d-ed74a2fe40e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:06.191161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c899e4-d801-4724-b4a2-bea5e8f03759 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:06.230476 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d9436-a4d3-44bf-9a03-bd1eb20f077b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:06.231533 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d9436-a4d3-44bf-9a03-bd1eb20f077b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:06.232029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d9436-a4d3-44bf-9a03-bd1eb20f077b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:06.232614 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d9436-a4d3-44bf-9a03-bd1eb20f077b None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:06.233575 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35d9436-a4d3-44bf-9a03-bd1eb20f077b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:06.234221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35d9436-a4d3-44bf-9a03-bd1eb20f077b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:06.235877 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35d9436-a4d3-44bf-9a03-bd1eb20f077b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:06.242617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 377/767] 10.222.0.41 () {56 vars in 1124 bytes} [Fri Jun 3 16:03:06 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:06.243756 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d9436-a4d3-44bf-9a03-bd1eb20f077b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:06.267324 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 391/768] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:03:06.267324 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec39826-27dc-49c6-955d-ed74a2fe40e3 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:06.274270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:06.343238 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:06.381611 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec39826-27dc-49c6-955d-ed74a2fe40e3 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:06.381952 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d9436-a4d3-44bf-9a03-bd1eb20f077b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:06.416741 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f25fac3-ce4a-4fea-a35b-ad336215f7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:06.416971 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba21ce83-543d-4919-9e73-4bf10d40af8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:06.417402 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21ce83-543d-4919-9e73-4bf10d40af8a None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:06.417572 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25fac3-ce4a-4fea-a35b-ad336215f7c5 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:06.417675 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21ce83-543d-4919-9e73-4bf10d40af8a None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:06.417834 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25fac3-ce4a-4fea-a35b-ad336215f7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:06.417946 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21ce83-543d-4919-9e73-4bf10d40af8a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:06.418091 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25fac3-ce4a-4fea-a35b-ad336215f7c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:06.418894 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba21ce83-543d-4919-9e73-4bf10d40af8a None admin] RBAC: Adding query filter params (interface=public, service_id=b9c3d048147d4969a3da27db01c93607, region_id=RegionOne) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:06.419210 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba21ce83-543d-4919-9e73-4bf10d40af8a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:06.420617 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba21ce83-543d-4919-9e73-4bf10d40af8a None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:06.429129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 392/769] 10.222.0.41 () {56 vars in 1251 bytes} [Fri Jun 3 16:03:06 2022] GET /identity/v3/endpoints?service_id=b9c3d048147d4969a3da27db01c93607&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:06.429731 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f25fac3-ce4a-4fea-a35b-ad336215f7c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:06.429929 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba21ce83-543d-4919-9e73-4bf10d40af8a None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:06.431358 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f25fac3-ce4a-4fea-a35b-ad336215f7c5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:06.450332 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-4f25fac3-ce4a-4fea-a35b-ad336215f7c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:06.450332 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:06.450332 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:06.450332 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:06.450332 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:06.450332 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:06.450332 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:06.450332 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:06.450332 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:06.450332 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:06.450332 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:06.450332 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:06.450332 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:03:06.450332 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:03:06.450332 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:03:06.451273 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:03:06.451273 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:06.451273 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:06.451273 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:06.451273 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:06.451273 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:06.451273 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:06.451273 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:06.451273 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:06.451273 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:06.451273 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:06.451273 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:06.451273 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:06.451273 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:06.451273 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:06.451273 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:06.451273 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:06.452191 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:03:06.452191 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:03:06.452191 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:06.452191 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:06.452191 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 378/770] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:06.452881 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f25fac3-ce4a-4fea-a35b-ad336215f7c5 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:06.538433 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba21ce83-543d-4919-9e73-4bf10d40af8a None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:06.555781 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:06.561490 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f25fac3-ce4a-4fea-a35b-ad336215f7c5 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:06.572582 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f0f54-8f56-4be2-b339-4e6394a0e8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:06.573382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f0f54-8f56-4be2-b339-4e6394a0e8f6 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:06.573665 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f0f54-8f56-4be2-b339-4e6394a0e8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:06.573943 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f0f54-8f56-4be2-b339-4e6394a0e8f6 None admin] PATH_INFO: `/v3/services/network` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:06.574589 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f0f54-8f56-4be2-b339-4e6394a0e8f6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:06.576004 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f0f54-8f56-4be2-b339-4e6394a0e8f6 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:06.578594 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:06.591386 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-162f0f54-8f56-4be2-b339-4e6394a0e8f6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 03 16:03:06.591386 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:06.591386 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 16:03:06.591386 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 16:03:06.591386 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 16:03:06.591386 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 16:03:06.591386 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 16:03:06.591386 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:06.591386 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 03 16:03:06.591386 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:06.591386 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 16:03:06.591386 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:06.591386 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:06.591386 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:06.591386 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:06.591386 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:06.591386 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:06.592433 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:06.592433 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:06.592433 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:06.592433 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:06.592433 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:06.592433 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:06.592433 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 16:03:06.592433 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 16:03:06.592433 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 16:03:06.592433 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 16:03:06.592433 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:06.592433 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:06.592433 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:06.592433 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:06.592433 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:06.592433 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:06.593257 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:06.593257 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:06.593257 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:06.593257 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:06.593257 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:06.593257 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:06.593257 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:06.593257 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:06.593257 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:06.593257 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:06.593257 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 16:03:06.593257 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:06.593257 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 03 16:03:06.593257 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:06.598156 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 393/771] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:03:06 2022] GET /identity/v3/services/network => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:06.599013 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f0f54-8f56-4be2-b339-4e6394a0e8f6 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:06.608920 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78560a2f-5489-4dc4-95a7-016c61ba2ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:06.609595 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78560a2f-5489-4dc4-95a7-016c61ba2ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:06.609873 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78560a2f-5489-4dc4-95a7-016c61ba2ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:06.610574 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78560a2f-5489-4dc4-95a7-016c61ba2ff5 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:06.611271 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78560a2f-5489-4dc4-95a7-016c61ba2ff5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:06.613040 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78560a2f-5489-4dc4-95a7-016c61ba2ff5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:06.619811 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 379/772] 10.222.0.41 () {56 vars in 1087 bytes} [Fri Jun 3 16:03:06 2022] GET /identity/v3/services => generated 772 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:06.620758 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78560a2f-5489-4dc4-95a7-016c61ba2ff5 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:06.805789 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f0f54-8f56-4be2-b339-4e6394a0e8f6 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:06.835580 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78560a2f-5489-4dc4-95a7-016c61ba2ff5 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:06.856982 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d4a79-c8c8-45aa-b108-951848dc017a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:06.857763 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d4a79-c8c8-45aa-b108-951848dc017a None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:06.858039 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d4a79-c8c8-45aa-b108-951848dc017a None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:06.858318 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d4a79-c8c8-45aa-b108-951848dc017a None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:06.859044 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465d4a79-c8c8-45aa-b108-951848dc017a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:06.859380 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465d4a79-c8c8-45aa-b108-951848dc017a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:06.860874 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465d4a79-c8c8-45aa-b108-951848dc017a None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:06.875990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5a221-3502-4dde-894f-aff64edb0830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:06.876647 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5a221-3502-4dde-894f-aff64edb0830 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:06.876933 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5a221-3502-4dde-894f-aff64edb0830 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:06.877212 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5a221-3502-4dde-894f-aff64edb0830 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:06.878452 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5a221-3502-4dde-894f-aff64edb0830 None admin] RBAC: Adding query filter params (name=network) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:06.878759 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5a221-3502-4dde-894f-aff64edb0830 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:06.879182 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 394/773] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:03:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:06.880088 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d4a79-c8c8-45aa-b108-951848dc017a None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:06.880191 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5a221-3502-4dde-894f-aff64edb0830 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:06.881464 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:06.889600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 380/774] 10.222.0.41 () {56 vars in 1112 bytes} [Fri Jun 3 16:03:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:06.895978 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5a221-3502-4dde-894f-aff64edb0830 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:06.971669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:07.054170 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5a221-3502-4dde-894f-aff64edb0830 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:07.092760 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d4a79-c8c8-45aa-b108-951848dc017a None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:07.093664 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0abd532-9e05-4621-85e3-595265a5653f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:07.094727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abd532-9e05-4621-85e3-595265a5653f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:07.095209 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abd532-9e05-4621-85e3-595265a5653f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:07.095707 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abd532-9e05-4621-85e3-595265a5653f None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:07.096668 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0abd532-9e05-4621-85e3-595265a5653f None admin] RBAC: Adding query filter params (type=network) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:07.097198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0abd532-9e05-4621-85e3-595265a5653f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:07.098773 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0abd532-9e05-4621-85e3-595265a5653f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:07.105220 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 381/775] 10.222.0.41 () {56 vars in 1112 bytes} [Fri Jun 3 16:03:06 2022] GET /identity/v3/services?type=network => generated 350 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:07.111549 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0abd532-9e05-4621-85e3-595265a5653f None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:07.179467 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f620a334-024c-414d-aea9-c9652a5739cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:07.180281 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620a334-024c-414d-aea9-c9652a5739cc None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:07.180568 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620a334-024c-414d-aea9-c9652a5739cc None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:07.180846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620a334-024c-414d-aea9-c9652a5739cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:07.187681 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:07.208556 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f620a334-024c-414d-aea9-c9652a5739cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:07.210019 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f620a334-024c-414d-aea9-c9652a5739cc None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:07.228665 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0abd532-9e05-4621-85e3-595265a5653f None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:07.241778 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-f620a334-024c-414d-aea9-c9652a5739cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:03:07.241778 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:07.241778 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:07.241778 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:07.241778 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:07.241778 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:07.241778 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:07.241778 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:07.241778 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:07.241778 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:07.241778 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:07.241778 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:07.241778 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:03:07.241778 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:03:07.241778 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:03:07.242702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:03:07.242702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:07.242702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:07.242702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:07.242702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:07.242702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:07.242702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:07.242702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:07.242702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:07.242702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:07.242702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:07.242702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:07.242702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:07.242702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:07.242702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:07.242702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:07.242702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:07.243642 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:03:07.243642 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:03:07.243642 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:03:07.243642 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:03:07.243642 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:03:07.243642 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:03:07.243642 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:03:07.243642 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:07.263405 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 395/776] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:03:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:03:07.264415 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f620a334-024c-414d-aea9-c9652a5739cc None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:07.268766 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9550e5ab-9b84-4647-8fb8-d28697e7973d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:07.269733 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550e5ab-9b84-4647-8fb8-d28697e7973d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:07.270276 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550e5ab-9b84-4647-8fb8-d28697e7973d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:07.270749 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550e5ab-9b84-4647-8fb8-d28697e7973d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:07.271818 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9550e5ab-9b84-4647-8fb8-d28697e7973d None admin] RBAC: Adding query filter params (interface=public, service_id=c0cfe86ebb774890809f2d15c4e69954, region_id=RegionOne) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:07.272349 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9550e5ab-9b84-4647-8fb8-d28697e7973d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:07.273935 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9550e5ab-9b84-4647-8fb8-d28697e7973d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:07.282810 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 382/777] 10.222.0.41 () {56 vars in 1251 bytes} [Fri Jun 3 16:03:07 2022] GET /identity/v3/endpoints?service_id=c0cfe86ebb774890809f2d15c4e69954&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:07.288322 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9550e5ab-9b84-4647-8fb8-d28697e7973d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:07.391862 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9550e5ab-9b84-4647-8fb8-d28697e7973d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:07.423681 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:07.429403 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f808ab-3151-4282-b216-9469cdaec316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:07.430440 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f808ab-3151-4282-b216-9469cdaec316 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:07.430952 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f808ab-3151-4282-b216-9469cdaec316 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:07.431474 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f808ab-3151-4282-b216-9469cdaec316 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:07.432377 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f808ab-3151-4282-b216-9469cdaec316 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:07.434088 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f808ab-3151-4282-b216-9469cdaec316 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:07.437895 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f620a334-024c-414d-aea9-c9652a5739cc None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:07.441435 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 383/778] 10.222.0.41 () {56 vars in 1087 bytes} [Fri Jun 3 16:03:07 2022] GET /identity/v3/services => generated 772 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:07.476276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:07.503537 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f23d60-50c3-48ad-85ab-107fcabbde52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:07.504286 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f23d60-50c3-48ad-85ab-107fcabbde52 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:07.504580 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f23d60-50c3-48ad-85ab-107fcabbde52 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:07.504860 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f23d60-50c3-48ad-85ab-107fcabbde52 None admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:07.505674 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f23d60-50c3-48ad-85ab-107fcabbde52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:07.505988 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f23d60-50c3-48ad-85ab-107fcabbde52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:07.507500 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f23d60-50c3-48ad-85ab-107fcabbde52 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:07.532039 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 396/779] 10.222.0.41 () {56 vars in 1148 bytes} [Fri Jun 3 16:03:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:07.538310 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f808ab-3151-4282-b216-9469cdaec316 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:07.688043 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f808ab-3151-4282-b216-9469cdaec316 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:07.724980 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded76c8b-0381-4fc1-8718-39a739efc9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:07.726018 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded76c8b-0381-4fc1-8718-39a739efc9bb None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:07.726523 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded76c8b-0381-4fc1-8718-39a739efc9bb None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:07.727079 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded76c8b-0381-4fc1-8718-39a739efc9bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:07.728180 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded76c8b-0381-4fc1-8718-39a739efc9bb None admin] RBAC: Adding query filter params (role.id=fe695aedbe8249a28c992a19264bc34b, scope.project.id=c274254e510a468e9acbecc6298301cc, user.id=875140cf51d9410da45f8c3805b26204) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:07.728705 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded76c8b-0381-4fc1-8718-39a739efc9bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:07.730328 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded76c8b-0381-4fc1-8718-39a739efc9bb None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:07.739669 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 384/780] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:03:07 2022] GET /identity/v3/role_assignments?user.id=875140cf51d9410da45f8c3805b26204&scope.project.id=c274254e510a468e9acbecc6298301cc&role.id=fe695aedbe8249a28c992a19264bc34b => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:07.851561 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe2689-eed6-4353-a4b8-f79b040eb0a1 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:07.851894 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe2689-eed6-4353-a4b8-f79b040eb0a1 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:07.852132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe2689-eed6-4353-a4b8-f79b040eb0a1 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:07.853891 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 397/781] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:07.859925 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3244a-28e9-4edc-b2fa-2019b51f0b8c None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:07.860868 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3244a-28e9-4edc-b2fa-2019b51f0b8c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:07.861353 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3244a-28e9-4edc-b2fa-2019b51f0b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:07.884164 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-00f3244a-28e9-4edc-b2fa-2019b51f0b8c None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:07.885436 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-00f3244a-28e9-4edc-b2fa-2019b51f0b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:07.957476 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:07.979421 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 385/782] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:03:08.001062 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe2689-eed6-4353-a4b8-f79b040eb0a1 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:08.114438 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe2689-eed6-4353-a4b8-f79b040eb0a1 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:08.149418 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc0bceb-0669-480c-b290-1bbd44158909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:08.150222 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0bceb-0669-480c-b290-1bbd44158909 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:08.150510 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0bceb-0669-480c-b290-1bbd44158909 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:08.150790 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0bceb-0669-480c-b290-1bbd44158909 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:08.162279 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc0bceb-0669-480c-b290-1bbd44158909 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:03:08.170163 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-4cc0bceb-0669-480c-b290-1bbd44158909 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:03:08.170163 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:08.170163 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:08.170163 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:08.170163 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:08.170163 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:08.170163 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:08.170163 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:08.170163 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:08.170163 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:08.170163 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:08.170163 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:08.170163 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:03:08.170163 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:03:08.170163 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:03:08.170163 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:03:08.171065 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:03:08.171065 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:03:08.171065 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:03:08.171065 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:03:08.171065 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:08.171065 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:08.171065 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:08.171065 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:08.171065 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:08.171065 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:08.171065 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:08.171065 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:08.171065 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:08.171065 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:08.171065 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:08.171065 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:08.171917 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:08.171917 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:08.171917 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:08.171917 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:08.171917 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:03:08.171917 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:03:08.171917 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:03:08.171917 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:03:08.171917 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:03:08.171917 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:03:08.171917 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:03:08.171917 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:08.174847 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 398/783] 10.222.0.41 () {56 vars in 1096 bytes} [Fri Jun 3 16:03:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:08.178001 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3244a-28e9-4edc-b2fa-2019b51f0b8c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:08.329081 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3244a-28e9-4edc-b2fa-2019b51f0b8c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:08.374164 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215115b3-954d-4402-bd4e-45c35bd2560d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:08.374886 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215115b3-954d-4402-bd4e-45c35bd2560d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:08.375174 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215115b3-954d-4402-bd4e-45c35bd2560d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:08.375468 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215115b3-954d-4402-bd4e-45c35bd2560d None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:08.376213 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215115b3-954d-4402-bd4e-45c35bd2560d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:08.376519 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215115b3-954d-4402-bd4e-45c35bd2560d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:08.378411 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215115b3-954d-4402-bd4e-45c35bd2560d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:08.391036 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 386/784] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:03:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:08.397405 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc0bceb-0669-480c-b290-1bbd44158909 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:08.441774 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7feee-79ce-40bb-96e2-529018aaba33 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:08.442095 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7feee-79ce-40bb-96e2-529018aaba33 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:08.442341 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7feee-79ce-40bb-96e2-529018aaba33 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:08.444116 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 387/785] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:08.452726 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897010d-1f6b-433e-a88c-43f80bf8ebde None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:08.453013 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897010d-1f6b-433e-a88c-43f80bf8ebde None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:08.453293 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897010d-1f6b-433e-a88c-43f80bf8ebde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:08.459078 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:08.489872 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-3897010d-1f6b-433e-a88c-43f80bf8ebde None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:08.491069 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3897010d-1f6b-433e-a88c-43f80bf8ebde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:08.578114 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc0bceb-0669-480c-b290-1bbd44158909 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:08.581790 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:08.622863 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 388/786] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:03:08.624018 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ec1c72-584b-49c6-aa1b-283c0c741f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:08.624819 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec1c72-584b-49c6-aa1b-283c0c741f5f None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:08.625103 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec1c72-584b-49c6-aa1b-283c0c741f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:08.625380 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec1c72-584b-49c6-aa1b-283c0c741f5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:08.644560 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ec1c72-584b-49c6-aa1b-283c0c741f5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:08.646572 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897010d-1f6b-433e-a88c-43f80bf8ebde None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:08.649877 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ec1c72-584b-49c6-aa1b-283c0c741f5f None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:08.686129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-d7ec1c72-584b-49c6-aa1b-283c0c741f5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:08.686129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:08.686129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:08.686129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:08.686129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:08.686129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:08.686129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:08.686129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:08.686129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:08.686129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:08.686129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:08.686129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:08.686129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:03:08.686129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:03:08.686129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:03:08.687088 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:03:08.687088 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:08.687088 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:08.687088 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:08.687088 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:08.687088 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:08.687088 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:08.687088 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:08.687088 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:08.687088 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:08.687088 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:08.687088 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:08.687088 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:08.687088 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:08.687088 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:08.687088 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:08.687088 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:08.687999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:03:08.687999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:03:08.687999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:08.687999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:08.691132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 399/787] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:08.696882 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ec1c72-584b-49c6-aa1b-283c0c741f5f None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:08.776279 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897010d-1f6b-433e-a88c-43f80bf8ebde None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:08.813003 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbaf985-9967-49cd-8332-5b6c78909d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:08.813800 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbaf985-9967-49cd-8332-5b6c78909d19 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:08.814085 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbaf985-9967-49cd-8332-5b6c78909d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:08.814365 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbaf985-9967-49cd-8332-5b6c78909d19 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:08.825382 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbaf985-9967-49cd-8332-5b6c78909d19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:03:08.833856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-0cbaf985-9967-49cd-8332-5b6c78909d19 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:03:08.833856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:08.833856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:08.833856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:08.833856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:08.833856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:08.833856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:08.833856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:08.833856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:08.833856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:08.833856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:08.833856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:08.833856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:03:08.833856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:03:08.833856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:03:08.833856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:03:08.836180 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:03:08.836180 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:03:08.836180 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:03:08.836180 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:03:08.836180 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:08.836180 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:08.836180 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:08.836180 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:08.836180 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:08.836180 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:08.836180 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:08.836180 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:08.836180 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:08.836180 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:08.836180 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:08.836180 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:08.838247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:08.838247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:08.838247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:08.838247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:08.838247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:03:08.838247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:03:08.838247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:03:08.838247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:03:08.838247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:03:08.838247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:03:08.838247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:03:08.838247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:08.838247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 389/788] 10.222.0.41 () {56 vars in 1096 bytes} [Fri Jun 3 16:03:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:03:08.847332 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbaf985-9967-49cd-8332-5b6c78909d19 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:08.883180 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:08.939672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:08.958037 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbaf985-9967-49cd-8332-5b6c78909d19 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:08.969178 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ec1c72-584b-49c6-aa1b-283c0c741f5f None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:08.996780 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632da54d-f128-4025-adc9-8cda8688491b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:08.997754 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632da54d-f128-4025-adc9-8cda8688491b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:08.998042 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632da54d-f128-4025-adc9-8cda8688491b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:08.998324 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632da54d-f128-4025-adc9-8cda8688491b None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:08.999079 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632da54d-f128-4025-adc9-8cda8688491b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:08.999419 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632da54d-f128-4025-adc9-8cda8688491b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:09.001341 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632da54d-f128-4025-adc9-8cda8688491b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:09.010786 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 390/789] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:03:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:09.016704 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632da54d-f128-4025-adc9-8cda8688491b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:09.044011 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f6cd2-cc9f-49ef-9db0-c0865f6ec18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:09.044673 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f6cd2-cc9f-49ef-9db0-c0865f6ec18e None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:09.044952 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f6cd2-cc9f-49ef-9db0-c0865f6ec18e None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:09.045228 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f6cd2-cc9f-49ef-9db0-c0865f6ec18e None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:09.045959 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786f6cd2-cc9f-49ef-9db0-c0865f6ec18e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:09.046264 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786f6cd2-cc9f-49ef-9db0-c0865f6ec18e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:09.047654 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786f6cd2-cc9f-49ef-9db0-c0865f6ec18e None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:09.073774 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 400/790] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:03:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:09.079265 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f6cd2-cc9f-49ef-9db0-c0865f6ec18e None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:09.124230 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:09.129200 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632da54d-f128-4025-adc9-8cda8688491b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:09.171601 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567016f0-c0d0-4524-afd1-8351053726fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:09.172400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567016f0-c0d0-4524-afd1-8351053726fc None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:09.172690 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567016f0-c0d0-4524-afd1-8351053726fc None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:09.173146 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567016f0-c0d0-4524-afd1-8351053726fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:09.199009 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567016f0-c0d0-4524-afd1-8351053726fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:09.200507 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567016f0-c0d0-4524-afd1-8351053726fc None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:09.230765 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-567016f0-c0d0-4524-afd1-8351053726fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:09.230765 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:09.230765 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:09.230765 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:09.230765 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:09.230765 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:09.230765 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:09.230765 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:09.230765 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:09.230765 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:09.230765 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:09.230765 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:09.230765 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:03:09.230765 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:03:09.230765 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:03:09.232132 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:03:09.232132 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:09.232132 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:09.232132 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:09.232132 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:09.232132 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:09.232132 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:09.232132 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:09.232132 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:09.232132 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:09.232132 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:09.232132 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:09.232132 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:09.232132 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:09.232132 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:09.232132 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:09.232132 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:09.233368 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:03:09.233368 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:03:09.233368 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:09.233368 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:09.233368 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 391/791] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:03:09.233916 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b817ee7-0358-4c22-8aae-1f8b8f40d1b0 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:09.234157 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b817ee7-0358-4c22-8aae-1f8b8f40d1b0 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:09.234437 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b817ee7-0358-4c22-8aae-1f8b8f40d1b0 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:09.239338 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 392/792] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:09.242923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8e075-1009-4860-acdd-ca839ac4d9b7 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:09.243214 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8e075-1009-4860-acdd-ca839ac4d9b7 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:09.243526 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8e075-1009-4860-acdd-ca839ac4d9b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:09.271184 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-dea8e075-1009-4860-acdd-ca839ac4d9b7 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:09.273242 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-dea8e075-1009-4860-acdd-ca839ac4d9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:09.287707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f6cd2-cc9f-49ef-9db0-c0865f6ec18e None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:09.330203 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c893c-80f4-45b9-a2b7-7cd6cbf53608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:09.331580 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c893c-80f4-45b9-a2b7-7cd6cbf53608 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:09.331867 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c893c-80f4-45b9-a2b7-7cd6cbf53608 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:09.332142 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c893c-80f4-45b9-a2b7-7cd6cbf53608 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:09.341644 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89c893c-80f4-45b9-a2b7-7cd6cbf53608 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:09.343553 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89c893c-80f4-45b9-a2b7-7cd6cbf53608 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:09.361707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-c89c893c-80f4-45b9-a2b7-7cd6cbf53608 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 03 16:03:09.361707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:09.361707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:09.361707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:09.361707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:09.361707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:09.361707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:09.361707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:09.361707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:09.361707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:09.361707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:09.361707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:09.361707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:03:09.361707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:03:09.361707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:03:09.361707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:03:09.362698 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:09.362698 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:09.362698 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:03:09.362698 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:03:09.362698 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:03:09.362698 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:03:09.362698 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:09.362698 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:09.362698 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:09.362698 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:09.362698 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:09.362698 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:09.362698 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:09.362698 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:09.362698 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:09.362698 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:09.362698 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:09.362698 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:09.368099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:09.368099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:09.368099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:03:09.368099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:03:09.368099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:03:09.368099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:03:09.368099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:03:09.368099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:03:09.368099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 03 16:03:09.368099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:09.368099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 401/793] 10.222.0.41 () {56 vars in 1096 bytes} [Fri Jun 3 16:03:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:09.368833 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c893c-80f4-45b9-a2b7-7cd6cbf53608 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:09.371367 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 393/794] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:03:09.373260 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8e075-1009-4860-acdd-ca839ac4d9b7 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:09.392732 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:09.480938 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:09.503970 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8e075-1009-4860-acdd-ca839ac4d9b7 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:09.513646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c893c-80f4-45b9-a2b7-7cd6cbf53608 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:09.545571 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc20dc-a399-4490-bb35-c7c5162c42ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:09.546387 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc20dc-a399-4490-bb35-c7c5162c42ed None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:09.546569 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc20dc-a399-4490-bb35-c7c5162c42ed None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:09.546800 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc20dc-a399-4490-bb35-c7c5162c42ed None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:09.547674 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bc20dc-a399-4490-bb35-c7c5162c42ed None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:09.547987 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bc20dc-a399-4490-bb35-c7c5162c42ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:09.549971 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bc20dc-a399-4490-bb35-c7c5162c42ed None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:09.575597 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 394/795] 10.222.0.41 () {56 vars in 1135 bytes} [Fri Jun 3 16:03:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:09.576091 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971982b8-ca1a-4424-9a73-fdc7979e5271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:09.577155 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971982b8-ca1a-4424-9a73-fdc7979e5271 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:09.577694 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971982b8-ca1a-4424-9a73-fdc7979e5271 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:09.578070 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc20dc-a399-4490-bb35-c7c5162c42ed None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:09.578512 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971982b8-ca1a-4424-9a73-fdc7979e5271 None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:09.579471 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971982b8-ca1a-4424-9a73-fdc7979e5271 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:09.580029 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971982b8-ca1a-4424-9a73-fdc7979e5271 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:09.582966 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971982b8-ca1a-4424-9a73-fdc7979e5271 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:09.599465 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 402/796] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:03:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:09.600578 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971982b8-ca1a-4424-9a73-fdc7979e5271 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:09.690593 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc20dc-a399-4490-bb35-c7c5162c42ed None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:09.699432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:09.727854 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba378a70-acb9-4c9a-9d62-97a01fa3288f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:09.728669 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba378a70-acb9-4c9a-9d62-97a01fa3288f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:09.728947 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba378a70-acb9-4c9a-9d62-97a01fa3288f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:09.729227 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba378a70-acb9-4c9a-9d62-97a01fa3288f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:09.729883 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba378a70-acb9-4c9a-9d62-97a01fa3288f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:09.731267 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba378a70-acb9-4c9a-9d62-97a01fa3288f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:09.744320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971982b8-ca1a-4424-9a73-fdc7979e5271 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:09.744505 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-ba378a70-acb9-4c9a-9d62-97a01fa3288f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 03 16:03:09.744505 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:09.744505 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 16:03:09.744505 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 16:03:09.744505 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 16:03:09.744505 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 16:03:09.744505 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 16:03:09.744505 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:09.744505 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 03 16:03:09.744505 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:09.744505 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 16:03:09.744505 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:09.744505 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:09.744505 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:09.744505 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:09.744505 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:09.745381 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:09.745381 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:09.745381 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:09.745381 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:09.745381 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:09.745381 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:09.745381 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:09.745381 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 16:03:09.745381 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 16:03:09.745381 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 16:03:09.745381 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 16:03:09.745381 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:09.745381 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:09.745381 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:09.745381 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:09.745381 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:09.746198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:09.746198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:09.746198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:09.746198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:09.746198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:09.746198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:09.746198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:09.746198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:09.746198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:09.746198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:09.746198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:09.746198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 16:03:09.746198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:09.746198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 03 16:03:09.746198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:09.750952 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 395/797] 10.222.0.41 () {56 vars in 1129 bytes} [Fri Jun 3 16:03:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:09.751661 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba378a70-acb9-4c9a-9d62-97a01fa3288f None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:09.754633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:09.796194 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d36999d-ae75-4bcb-84e0-82206031d495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:09.797798 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36999d-ae75-4bcb-84e0-82206031d495 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:09.798691 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36999d-ae75-4bcb-84e0-82206031d495 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:09.799882 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36999d-ae75-4bcb-84e0-82206031d495 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:09.813788 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d36999d-ae75-4bcb-84e0-82206031d495 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:09.815423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d36999d-ae75-4bcb-84e0-82206031d495 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:09.833570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-5d36999d-ae75-4bcb-84e0-82206031d495 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:09.833570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:09.833570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:09.833570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:09.833570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:09.833570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:09.833570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:09.833570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:09.833570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:09.833570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:09.833570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:09.833570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:09.833570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:03:09.833570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:03:09.833570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:03:09.834499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:03:09.834499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:09.834499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:09.834499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:09.834499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:09.834499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:09.834499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:09.834499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:09.834499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:09.834499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:09.834499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:09.834499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:09.834499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:09.834499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:09.834499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:09.834499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:09.834499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:09.835431 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:03:09.835431 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:03:09.835431 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:09.835431 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:09.839480 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 403/798] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:09.840473 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d36999d-ae75-4bcb-84e0-82206031d495 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:09.879516 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba378a70-acb9-4c9a-9d62-97a01fa3288f None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:09.926632 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb80294-4e5e-4fb6-835f-1c1d70b40138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:09.928815 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb80294-4e5e-4fb6-835f-1c1d70b40138 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:09.929254 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb80294-4e5e-4fb6-835f-1c1d70b40138 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:09.929777 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb80294-4e5e-4fb6-835f-1c1d70b40138 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:09.941181 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb80294-4e5e-4fb6-835f-1c1d70b40138 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:09.943385 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb80294-4e5e-4fb6-835f-1c1d70b40138 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:09.947270 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d36999d-ae75-4bcb-84e0-82206031d495 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:09.961842 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-eeb80294-4e5e-4fb6-835f-1c1d70b40138 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 03 16:03:09.961842 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:09.961842 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:09.961842 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:09.961842 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:09.961842 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:09.961842 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:09.961842 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:09.961842 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:09.961842 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:09.961842 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:09.961842 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:09.961842 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:03:09.961842 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:03:09.961842 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:03:09.961842 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:03:09.963020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:09.963020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:09.963020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:03:09.963020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:03:09.963020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:03:09.963020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:03:09.963020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:09.963020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:09.963020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:09.963020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:09.963020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:09.963020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:09.963020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:09.963020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:09.963020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:09.963020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:09.963020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:09.963020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:09.964201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:09.964201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:09.964201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:03:09.964201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:03:09.964201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:03:09.964201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:03:09.964201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:03:09.964201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:03:09.964201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 03 16:03:09.964201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:09.972581 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 396/799] 10.222.0.41 () {56 vars in 1093 bytes} [Fri Jun 3 16:03:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:09.973720 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb80294-4e5e-4fb6-835f-1c1d70b40138 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:09.989566 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:10.003889 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c944fcd-bf19-4c0b-93ce-31b2ac57e0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:10.004890 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c944fcd-bf19-4c0b-93ce-31b2ac57e0e9 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:10.005360 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c944fcd-bf19-4c0b-93ce-31b2ac57e0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:10.005785 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c944fcd-bf19-4c0b-93ce-31b2ac57e0e9 None admin] PATH_INFO: `/v3/services` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:10.006664 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c944fcd-bf19-4c0b-93ce-31b2ac57e0e9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:10.007169 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c944fcd-bf19-4c0b-93ce-31b2ac57e0e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:10.008793 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c944fcd-bf19-4c0b-93ce-31b2ac57e0e9 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:10.014640 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 404/800] 10.222.0.41 () {56 vars in 1124 bytes} [Fri Jun 3 16:03:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:10.015862 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c944fcd-bf19-4c0b-93ce-31b2ac57e0e9 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:10.054333 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:10.099509 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb80294-4e5e-4fb6-835f-1c1d70b40138 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:10.120299 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c944fcd-bf19-4c0b-93ce-31b2ac57e0e9 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:10.136395 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d028dec-b984-4b27-b229-f15bc2f3ee02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:10.137068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d028dec-b984-4b27-b229-f15bc2f3ee02 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:10.137342 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d028dec-b984-4b27-b229-f15bc2f3ee02 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:10.137626 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d028dec-b984-4b27-b229-f15bc2f3ee02 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:10.138354 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d028dec-b984-4b27-b229-f15bc2f3ee02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:10.138666 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d028dec-b984-4b27-b229-f15bc2f3ee02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:10.140404 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d028dec-b984-4b27-b229-f15bc2f3ee02 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:10.158155 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744aed60-01d6-4165-b378-f7f205bd1e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:10.159488 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 397/801] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:03:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:10.160200 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744aed60-01d6-4165-b378-f7f205bd1e7c None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:10.160425 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d028dec-b984-4b27-b229-f15bc2f3ee02 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:10.161000 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744aed60-01d6-4165-b378-f7f205bd1e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:10.161484 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744aed60-01d6-4165-b378-f7f205bd1e7c None admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:10.162531 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744aed60-01d6-4165-b378-f7f205bd1e7c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:10.163045 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744aed60-01d6-4165-b378-f7f205bd1e7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:10.164738 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744aed60-01d6-4165-b378-f7f205bd1e7c None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:10.183923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 405/802] 10.222.0.41 () {56 vars in 1133 bytes} [Fri Jun 3 16:03:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:10.185133 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744aed60-01d6-4165-b378-f7f205bd1e7c None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:10.235872 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:10.281694 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d028dec-b984-4b27-b229-f15bc2f3ee02 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:10.296937 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744aed60-01d6-4165-b378-f7f205bd1e7c None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:10.307196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:10.320099 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4ec34-61ce-434b-ad23-5d5ad994d2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:10.320761 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4ec34-61ce-434b-ad23-5d5ad994d2a5 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:10.321047 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4ec34-61ce-434b-ad23-5d5ad994d2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:10.321326 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4ec34-61ce-434b-ad23-5d5ad994d2a5 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:10.322067 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a4ec34-61ce-434b-ad23-5d5ad994d2a5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:10.322373 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a4ec34-61ce-434b-ad23-5d5ad994d2a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:10.323828 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a4ec34-61ce-434b-ad23-5d5ad994d2a5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:10.328580 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 398/803] 10.222.0.41 () {56 vars in 1124 bytes} [Fri Jun 3 16:03:10 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:10.329343 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4ec34-61ce-434b-ad23-5d5ad994d2a5 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:10.334840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5323978-eaed-48d4-bdc1-8fc613efcc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:10.335656 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5323978-eaed-48d4-bdc1-8fc613efcc9c None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:10.335940 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5323978-eaed-48d4-bdc1-8fc613efcc9c None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:10.336213 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5323978-eaed-48d4-bdc1-8fc613efcc9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:10.342942 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5323978-eaed-48d4-bdc1-8fc613efcc9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:10.344392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5323978-eaed-48d4-bdc1-8fc613efcc9c None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:10.358509 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-d5323978-eaed-48d4-bdc1-8fc613efcc9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:03:10.358509 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:10.358509 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:10.358509 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:10.358509 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:10.358509 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:10.358509 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:10.358509 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:10.358509 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:10.358509 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:10.358509 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:10.358509 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:10.358509 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:03:10.358509 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:03:10.358509 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:03:10.359382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:03:10.359382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:10.359382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:10.359382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:10.359382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:10.359382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:10.359382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:10.359382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:10.359382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:10.359382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:10.359382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:10.359382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:10.359382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:10.359382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:10.359382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:10.359382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:10.359382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:10.360285 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:03:10.360285 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:03:10.360285 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:03:10.360285 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:03:10.360285 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:03:10.360285 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:03:10.360285 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:03:10.360285 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:10.360285 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 406/804] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:03:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:10.361818 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f271903-266f-4c0c-8575-786b4e74bfd8 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:10.361818 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f271903-266f-4c0c-8575-786b4e74bfd8 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:10.361818 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f271903-266f-4c0c-8575-786b4e74bfd8 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:10.362931 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 407/805] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:10.364041 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341a1d4-20e9-4cbf-94b5-3712f55754e9 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:10.364336 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341a1d4-20e9-4cbf-94b5-3712f55754e9 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:10.364673 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341a1d4-20e9-4cbf-94b5-3712f55754e9 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:10.365871 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 408/806] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:10.366607 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2341a1d4-20e9-4cbf-94b5-3712f55754e9 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:10.432319 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4ec34-61ce-434b-ad23-5d5ad994d2a5 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:10.465715 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbabc0a7-74f2-4871-8a19-d5370565f0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:10.465913 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2341a1d4-20e9-4cbf-94b5-3712f55754e9 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:10.466349 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbabc0a7-74f2-4871-8a19-d5370565f0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:10.466636 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbabc0a7-74f2-4871-8a19-d5370565f0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:10.466917 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbabc0a7-74f2-4871-8a19-d5370565f0d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:10.472595 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:10.499023 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4378bb-560c-49cc-b8bd-ff7350804662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:10.499631 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4378bb-560c-49cc-b8bd-ff7350804662 None admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:10.499915 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4378bb-560c-49cc-b8bd-ff7350804662 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:10.500189 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4378bb-560c-49cc-b8bd-ff7350804662 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:10.500916 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4378bb-560c-49cc-b8bd-ff7350804662 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:10.502788 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4378bb-560c-49cc-b8bd-ff7350804662 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:10.521326 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:10.522368 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 409/807] 10.222.0.41 () {60 vars in 1144 bytes} [Fri Jun 3 16:03:10 2022] POST /identity/v3/endpoints => generated 346 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:03:10.523187 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4378bb-560c-49cc-b8bd-ff7350804662 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:10.541591 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbabc0a7-74f2-4871-8a19-d5370565f0d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:10.543111 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbabc0a7-74f2-4871-8a19-d5370565f0d1 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:10.648108 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4378bb-560c-49cc-b8bd-ff7350804662 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:10.656196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 399/808] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:03:10 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 16:03:10.657523 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e7f1b8-efd9-4004-b595-a5c29f0bad08 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:10.657808 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e7f1b8-efd9-4004-b595-a5c29f0bad08 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:10.658087 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e7f1b8-efd9-4004-b595-a5c29f0bad08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:10.675779 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-21e7f1b8-efd9-4004-b595-a5c29f0bad08 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:10.677007 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-21e7f1b8-efd9-4004-b595-a5c29f0bad08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:10.692903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa7a2e6-8f6f-4647-8b1e-b6105bac0818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:10.693587 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7a2e6-8f6f-4647-8b1e-b6105bac0818 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:10.693870 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7a2e6-8f6f-4647-8b1e-b6105bac0818 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:10.694146 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7a2e6-8f6f-4647-8b1e-b6105bac0818 None admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:10.695067 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa7a2e6-8f6f-4647-8b1e-b6105bac0818 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:10.695407 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa7a2e6-8f6f-4647-8b1e-b6105bac0818 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:10.696973 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa7a2e6-8f6f-4647-8b1e-b6105bac0818 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:10.713104 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 410/809] 10.222.0.41 () {56 vars in 1148 bytes} [Fri Jun 3 16:03:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:10.713307 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31a9b0-1a16-49b0-b61d-d00b3e5eae67 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:10.713567 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31a9b0-1a16-49b0-b61d-d00b3e5eae67 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:10.713775 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31a9b0-1a16-49b0-b61d-d00b3e5eae67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:10.731887 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-9f31a9b0-1a16-49b0-b61d-d00b3e5eae67 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:10.733163 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-9f31a9b0-1a16-49b0-b61d-d00b3e5eae67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:10.747354 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:10.763675 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 400/810] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:03:10.764497 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e7f1b8-efd9-4004-b595-a5c29f0bad08 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:10.787691 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:10.812191 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 411/811] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:03:10.813011 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f31a9b0-1a16-49b0-b61d-d00b3e5eae67 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:10.882804 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e7f1b8-efd9-4004-b595-a5c29f0bad08 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:10.919443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae916b0-f071-4476-a31a-ce701c3e74d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:10.920543 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae916b0-f071-4476-a31a-ce701c3e74d7 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:10.921126 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae916b0-f071-4476-a31a-ce701c3e74d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:10.921413 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae916b0-f071-4476-a31a-ce701c3e74d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:10.925630 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f31a9b0-1a16-49b0-b61d-d00b3e5eae67 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:10.933826 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae916b0-f071-4476-a31a-ce701c3e74d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:10.935682 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae916b0-f071-4476-a31a-ce701c3e74d7 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:10.958580 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-0ae916b0-f071-4476-a31a-ce701c3e74d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:10.958580 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:10.958580 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:10.958580 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:10.958580 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:10.958580 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:10.958580 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:10.958580 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:10.958580 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:10.958580 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:10.958580 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:10.958580 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:10.958580 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:03:10.958580 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:03:10.958580 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:03:10.959986 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:03:10.959986 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:10.959986 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:10.959986 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:10.959986 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:10.959986 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:10.959986 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:10.959986 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:10.959986 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:10.959986 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:10.959986 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:10.959986 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:10.959986 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:10.959986 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:10.959986 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:10.959986 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:10.959986 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:10.960907 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:03:10.960907 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:03:10.960907 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:10.960907 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:10.961141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db58436d-2aa7-4167-829a-a2904f58b885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:10.961141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db58436d-2aa7-4167-829a-a2904f58b885 None admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:10.961616 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db58436d-2aa7-4167-829a-a2904f58b885 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:10.961616 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db58436d-2aa7-4167-829a-a2904f58b885 None admin] PATH_INFO: `/v3/projects/c274254e510a468e9acbecc6298301cc/users/6c5f513b781a4a73b443cded13359eda/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:10.963637 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db58436d-2aa7-4167-829a-a2904f58b885 None admin] RBAC: Authorizing `identity:create_grant(project_id=c274254e510a468e9acbecc6298301cc, user_id=6c5f513b781a4a73b443cded13359eda, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:10.964367 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 401/812] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:10.964980 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae916b0-f071-4476-a31a-ce701c3e74d7 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:10.965148 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db58436d-2aa7-4167-829a-a2904f58b885 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:10.987951 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 412/813] 10.222.0.41 () {58 vars in 1439 bytes} [Fri Jun 3 16:03:10 2022] PUT /identity/v3/projects/c274254e510a468e9acbecc6298301cc/users/6c5f513b781a4a73b443cded13359eda/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:03:10.987951 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db58436d-2aa7-4167-829a-a2904f58b885 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:10.997036 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:11.074255 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:11.118907 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db58436d-2aa7-4167-829a-a2904f58b885 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:11.123356 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae916b0-f071-4476-a31a-ce701c3e74d7 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:11.162082 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4199e73-edbf-40f8-87cb-d3bb2956c663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:11.163151 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4199e73-edbf-40f8-87cb-d3bb2956c663 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:11.163666 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4199e73-edbf-40f8-87cb-d3bb2956c663 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:11.164047 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4199e73-edbf-40f8-87cb-d3bb2956c663 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:11.169151 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5d275-5827-452c-be1b-331cc5bd43ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:11.170249 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5d275-5827-452c-be1b-331cc5bd43ac None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:11.170535 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5d275-5827-452c-be1b-331cc5bd43ac None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:11.170812 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5d275-5827-452c-be1b-331cc5bd43ac None admin] PATH_INFO: `/v3/services/network` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:11.171509 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba5d275-5827-452c-be1b-331cc5bd43ac None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:11.173238 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba5d275-5827-452c-be1b-331cc5bd43ac None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:11.177374 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4199e73-edbf-40f8-87cb-d3bb2956c663 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:03:11.186665 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-d4199e73-edbf-40f8-87cb-d3bb2956c663 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:03:11.186665 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:11.186665 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:11.186665 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:11.186665 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:11.186665 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:11.186665 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:11.186665 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:11.186665 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:11.186665 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:11.186665 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:11.186665 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:11.186665 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:03:11.186665 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:03:11.186665 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:03:11.186665 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:03:11.188330 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:03:11.188330 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:03:11.188330 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:03:11.188330 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:03:11.188330 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:11.188330 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:11.188330 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:11.188330 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:11.188330 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:11.188330 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:11.188330 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:11.188330 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:11.188330 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:11.188330 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:11.188330 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:11.188330 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:11.189837 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:11.189837 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:11.189837 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:11.189837 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:11.189837 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:03:11.189837 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:03:11.189837 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:03:11.189837 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:03:11.189837 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:03:11.189837 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:03:11.189837 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:03:11.189837 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:11.190929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-9ba5d275-5827-452c-be1b-331cc5bd43ac None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 03 16:03:11.190929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:11.190929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 16:03:11.190929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 16:03:11.190929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 16:03:11.190929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 16:03:11.190929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 16:03:11.190929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:11.190929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 03 16:03:11.190929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:11.190929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 16:03:11.190929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:11.190929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:11.190929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:11.190929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:11.190929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:11.190929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:11.192499 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:11.192499 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:11.192499 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:11.192499 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:11.192499 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:11.192499 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:11.192499 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 16:03:11.192499 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 16:03:11.192499 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 16:03:11.192499 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 16:03:11.192499 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:11.192499 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:11.192499 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:11.192499 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:11.192499 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:11.192499 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:11.195123 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:11.195123 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:11.195123 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:11.195123 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:11.195123 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:11.195123 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:11.195123 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:11.195123 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:11.195123 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:11.195123 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:11.195123 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 16:03:11.195123 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:11.195123 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 03 16:03:11.195123 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:11.195123 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 402/814] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:03:10 2022] GET /identity/v3/services/network => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:11.197251 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 413/815] 10.222.0.41 () {56 vars in 1102 bytes} [Fri Jun 3 16:03:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:11.197413 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5d275-5827-452c-be1b-331cc5bd43ac None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:11.197574 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4199e73-edbf-40f8-87cb-d3bb2956c663 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:11.295759 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:11.319999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4199e73-edbf-40f8-87cb-d3bb2956c663 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:11.333668 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5d275-5827-452c-be1b-331cc5bd43ac None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:11.365189 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:11.368675 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adee5ae-f5d8-409c-b27a-355e0e6019c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:11.369511 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adee5ae-f5d8-409c-b27a-355e0e6019c2 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:11.369861 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adee5ae-f5d8-409c-b27a-355e0e6019c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:11.370068 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adee5ae-f5d8-409c-b27a-355e0e6019c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:11.370816 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adee5ae-f5d8-409c-b27a-355e0e6019c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:11.371212 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adee5ae-f5d8-409c-b27a-355e0e6019c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:11.371698 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e3474c-b887-4854-85e0-19cd59fba9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:11.372451 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3474c-b887-4854-85e0-19cd59fba9a0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:11.372603 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adee5ae-f5d8-409c-b27a-355e0e6019c2 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:11.372916 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3474c-b887-4854-85e0-19cd59fba9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:11.373292 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3474c-b887-4854-85e0-19cd59fba9a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:11.374117 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e3474c-b887-4854-85e0-19cd59fba9a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:11.374527 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e3474c-b887-4854-85e0-19cd59fba9a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:11.376033 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e3474c-b887-4854-85e0-19cd59fba9a0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:11.382357 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 414/816] 10.222.0.41 () {56 vars in 1103 bytes} [Fri Jun 3 16:03:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:11.383183 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adee5ae-f5d8-409c-b27a-355e0e6019c2 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:11.394919 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 403/817] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:03:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:11.395796 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e3474c-b887-4854-85e0-19cd59fba9a0 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:11.510995 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e3474c-b887-4854-85e0-19cd59fba9a0 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:11.511676 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adee5ae-f5d8-409c-b27a-355e0e6019c2 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:11.542228 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:11.549722 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d8d15c-b2b1-4134-aeaa-6365b0cf53bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:11.550527 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d8d15c-b2b1-4134-aeaa-6365b0cf53bc None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:11.550913 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d8d15c-b2b1-4134-aeaa-6365b0cf53bc None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:11.551309 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d8d15c-b2b1-4134-aeaa-6365b0cf53bc None admin] PATH_INFO: `/v3/services` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:11.552124 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d8d15c-b2b1-4134-aeaa-6365b0cf53bc None admin] RBAC: Adding query filter params (name=network) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:11.552531 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d8d15c-b2b1-4134-aeaa-6365b0cf53bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:11.553986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d8d15c-b2b1-4134-aeaa-6365b0cf53bc None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:11.557687 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31294a5c-d8fa-44a1-b514-a696e49e46d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:11.558652 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31294a5c-d8fa-44a1-b514-a696e49e46d0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:11.559082 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31294a5c-d8fa-44a1-b514-a696e49e46d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:11.559523 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 415/818] 10.222.0.41 () {56 vars in 1112 bytes} [Fri Jun 3 16:03:11 2022] GET /identity/v3/services?name=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:11.559675 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31294a5c-d8fa-44a1-b514-a696e49e46d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:11.560532 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d8d15c-b2b1-4134-aeaa-6365b0cf53bc None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:11.574955 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31294a5c-d8fa-44a1-b514-a696e49e46d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:11.576656 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31294a5c-d8fa-44a1-b514-a696e49e46d0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:11.598639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-31294a5c-d8fa-44a1-b514-a696e49e46d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:11.598639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:11.598639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:11.598639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:11.598639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:11.598639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:11.598639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:11.598639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:11.598639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:11.598639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:11.598639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:11.598639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:11.598639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:03:11.598639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:03:11.598639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:03:11.599728 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:03:11.599728 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:11.599728 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:11.599728 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:11.599728 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:11.599728 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:11.599728 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:11.599728 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:11.599728 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:11.599728 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:11.599728 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:11.599728 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:11.599728 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:11.599728 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:11.599728 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:11.599728 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:11.599728 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:11.600811 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:03:11.600811 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:03:11.600811 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:11.600811 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:11.606773 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 404/819] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:11.607762 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31294a5c-d8fa-44a1-b514-a696e49e46d0 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:11.661252 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:11.708447 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d8d15c-b2b1-4134-aeaa-6365b0cf53bc None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:11.729448 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31294a5c-d8fa-44a1-b514-a696e49e46d0 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:11.745535 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ea502-7070-4815-b73e-125ba0065251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:11.746491 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ea502-7070-4815-b73e-125ba0065251 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:11.746904 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ea502-7070-4815-b73e-125ba0065251 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:11.747280 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ea502-7070-4815-b73e-125ba0065251 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:11.754838 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64ea502-7070-4815-b73e-125ba0065251 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:11.756547 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64ea502-7070-4815-b73e-125ba0065251 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:11.771149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675bfac2-0ea5-49a3-9548-7abeb64c9062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:11.772052 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675bfac2-0ea5-49a3-9548-7abeb64c9062 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:11.772576 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675bfac2-0ea5-49a3-9548-7abeb64c9062 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:11.772996 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675bfac2-0ea5-49a3-9548-7abeb64c9062 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:11.773786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-c64ea502-7070-4815-b73e-125ba0065251 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:03:11.773786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:11.773786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:11.773786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:11.773786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:11.773786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:11.773786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:11.773786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:11.773786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:11.773786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:11.773786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:11.773786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:11.773786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:03:11.773786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:03:11.773786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:03:11.774799 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:03:11.774799 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:11.774799 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:11.774799 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:11.774799 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:11.774799 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:11.774799 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:11.774799 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:11.774799 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:11.774799 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:11.774799 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:11.774799 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:11.774799 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:11.774799 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:11.774799 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:11.774799 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:11.774799 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:11.775869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:03:11.775869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:03:11.775869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:03:11.775869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:03:11.775869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:03:11.775869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:03:11.775869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:03:11.775869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:11.776481 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675bfac2-0ea5-49a3-9548-7abeb64c9062 None admin] RBAC: Adding query filter params (type=network) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:11.776934 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675bfac2-0ea5-49a3-9548-7abeb64c9062 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:11.782546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 416/820] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:03:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:11.782546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ea502-7070-4815-b73e-125ba0065251 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:11.782766 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675bfac2-0ea5-49a3-9548-7abeb64c9062 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:11.788868 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 405/821] 10.222.0.41 () {56 vars in 1112 bytes} [Fri Jun 3 16:03:11 2022] GET /identity/v3/services?type=network => generated 350 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:11.791363 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675bfac2-0ea5-49a3-9548-7abeb64c9062 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:11.846985 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:11.902829 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ea502-7070-4815-b73e-125ba0065251 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:11.910646 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675bfac2-0ea5-49a3-9548-7abeb64c9062 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:11.920645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:11.948157 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286376d-0a82-439b-8d51-81d703fd6db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:11.949137 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58431d6-3ee6-4f24-8713-3a959049999a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:11.949662 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286376d-0a82-439b-8d51-81d703fd6db4 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:11.949914 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58431d6-3ee6-4f24-8713-3a959049999a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:11.950271 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58431d6-3ee6-4f24-8713-3a959049999a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:11.950499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286376d-0a82-439b-8d51-81d703fd6db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:11.950817 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58431d6-3ee6-4f24-8713-3a959049999a None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:11.951153 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286376d-0a82-439b-8d51-81d703fd6db4 None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:11.951782 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58431d6-3ee6-4f24-8713-3a959049999a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:11.952172 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58431d6-3ee6-4f24-8713-3a959049999a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:11.952441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286376d-0a82-439b-8d51-81d703fd6db4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:11.952910 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286376d-0a82-439b-8d51-81d703fd6db4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:11.953808 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58431d6-3ee6-4f24-8713-3a959049999a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:11.954670 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286376d-0a82-439b-8d51-81d703fd6db4 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:11.971922 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 406/822] 10.222.0.41 () {56 vars in 1148 bytes} [Fri Jun 3 16:03:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:11.971922 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58431d6-3ee6-4f24-8713-3a959049999a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:11.975616 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 417/823] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:03:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:11.976756 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286376d-0a82-439b-8d51-81d703fd6db4 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:12.092684 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286376d-0a82-439b-8d51-81d703fd6db4 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:12.098810 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58431d6-3ee6-4f24-8713-3a959049999a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:12.108122 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:12.137805 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec28ad9d-83f0-46ce-a4da-3621118d5342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:12.138465 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec28ad9d-83f0-46ce-a4da-3621118d5342 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:12.138752 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec28ad9d-83f0-46ce-a4da-3621118d5342 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:12.139030 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec28ad9d-83f0-46ce-a4da-3621118d5342 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:12.139882 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec28ad9d-83f0-46ce-a4da-3621118d5342 None admin] RBAC: Adding query filter params (role.id=75936a608b8149099ff54e317f3a6740, scope.project.id=c274254e510a468e9acbecc6298301cc, user.id=2fd08ea39ee94b92ac6763608e69f158) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:12.140200 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec28ad9d-83f0-46ce-a4da-3621118d5342 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:12.141630 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec28ad9d-83f0-46ce-a4da-3621118d5342 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:12.143655 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b05395-e4ba-48c7-8fcd-a2bdd9cb0df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:12.144541 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b05395-e4ba-48c7-8fcd-a2bdd9cb0df5 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:12.145028 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b05395-e4ba-48c7-8fcd-a2bdd9cb0df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:12.145507 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b05395-e4ba-48c7-8fcd-a2bdd9cb0df5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:12.146451 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b05395-e4ba-48c7-8fcd-a2bdd9cb0df5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:12.148059 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b05395-e4ba-48c7-8fcd-a2bdd9cb0df5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:12.159393 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 418/824] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:03:11 2022] GET /identity/v3/role_assignments?user.id=2fd08ea39ee94b92ac6763608e69f158&scope.project.id=c274254e510a468e9acbecc6298301cc&role.id=75936a608b8149099ff54e317f3a6740 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:12.159393 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec28ad9d-83f0-46ce-a4da-3621118d5342 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:12.190344 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 407/825] 10.222.0.41 () {60 vars in 1144 bytes} [Fri Jun 3 16:03:11 2022] POST /identity/v3/endpoints => generated 327 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:03:12.294498 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec28ad9d-83f0-46ce-a4da-3621118d5342 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:12.355388 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c0a2d-e283-4d51-ba66-1562800009b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:12.356227 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c0a2d-e283-4d51-ba66-1562800009b3 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:12.356512 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c0a2d-e283-4d51-ba66-1562800009b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:12.356787 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c0a2d-e283-4d51-ba66-1562800009b3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:12.368777 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596c0a2d-e283-4d51-ba66-1562800009b3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:12.370252 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596c0a2d-e283-4d51-ba66-1562800009b3 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:12.388179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-596c0a2d-e283-4d51-ba66-1562800009b3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 03 16:03:12.388179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:12.388179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:12.388179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:12.388179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:12.388179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:12.388179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:12.388179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:12.388179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:12.388179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:12.388179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:12.388179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:12.388179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:03:12.388179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:03:12.388179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:03:12.388179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:03:12.389178 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:12.389178 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:12.389178 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:03:12.389178 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:03:12.389178 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:03:12.389178 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:03:12.389178 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:12.389178 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:12.389178 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:12.389178 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:12.389178 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:12.389178 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:12.389178 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:12.389178 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:12.389178 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:12.389178 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:12.389178 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:12.389178 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:12.390096 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:12.390096 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:12.390096 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:03:12.390096 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:03:12.390096 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:03:12.390096 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:03:12.390096 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:03:12.390096 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:03:12.390096 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 03 16:03:12.390096 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:12.390096 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 419/826] 10.222.0.41 () {56 vars in 1117 bytes} [Fri Jun 3 16:03:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:12.399118 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b05395-e4ba-48c7-8fcd-a2bdd9cb0df5 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:12.450295 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:12.550797 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b05395-e4ba-48c7-8fcd-a2bdd9cb0df5 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:12.587977 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b89e40-3a66-42b4-851d-d44787ac926a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:12.588650 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b89e40-3a66-42b4-851d-d44787ac926a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:12.588934 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b89e40-3a66-42b4-851d-d44787ac926a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:12.589213 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b89e40-3a66-42b4-851d-d44787ac926a None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:12.590064 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b89e40-3a66-42b4-851d-d44787ac926a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:12.590459 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b89e40-3a66-42b4-851d-d44787ac926a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:12.591956 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b89e40-3a66-42b4-851d-d44787ac926a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:12.613319 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 408/827] 10.222.0.41 () {56 vars in 1149 bytes} [Fri Jun 3 16:03:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:12.618960 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c0a2d-e283-4d51-ba66-1562800009b3 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:12.660485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:12.741469 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c0a2d-e283-4d51-ba66-1562800009b3 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:12.779469 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6746c0db-2fa9-4c32-b7b7-fd5763c05c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:12.780273 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6746c0db-2fa9-4c32-b7b7-fd5763c05c6d None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:12.780558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6746c0db-2fa9-4c32-b7b7-fd5763c05c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:12.780836 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6746c0db-2fa9-4c32-b7b7-fd5763c05c6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:12.794327 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6746c0db-2fa9-4c32-b7b7-fd5763c05c6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:12.795817 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6746c0db-2fa9-4c32-b7b7-fd5763c05c6d None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:12.816170 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-6746c0db-2fa9-4c32-b7b7-fd5763c05c6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:12.816170 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:12.816170 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:12.816170 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:12.816170 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:12.816170 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:12.816170 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:12.816170 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:12.816170 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:12.816170 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:12.816170 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:12.816170 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:12.816170 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:03:12.816170 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:03:12.816170 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:03:12.817092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:03:12.817092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:12.817092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:12.817092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:12.817092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:12.817092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:12.817092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:12.817092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:12.817092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:12.817092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:12.817092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:12.817092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:12.817092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:12.817092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:12.817092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:12.817092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:12.817092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:12.817964 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:03:12.817964 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:03:12.817964 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:12.817964 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:12.818973 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 420/828] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:12.825238 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b89e40-3a66-42b4-851d-d44787ac926a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:12.855814 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132db8eb-8b54-483e-b9d4-50e32cc28ca2 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:12.856160 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132db8eb-8b54-483e-b9d4-50e32cc28ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:12.856374 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132db8eb-8b54-483e-b9d4-50e32cc28ca2 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:12.859340 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 421/829] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 16:03:12.864160 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3955e-9524-4389-bc20-957a5a5d9430 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:12.864448 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3955e-9524-4389-bc20-957a5a5d9430 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:12.864735 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3955e-9524-4389-bc20-957a5a5d9430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:12.889819 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-c5e3955e-9524-4389-bc20-957a5a5d9430 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:12.891749 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-c5e3955e-9524-4389-bc20-957a5a5d9430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:12.926272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:12.951159 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b89e40-3a66-42b4-851d-d44787ac926a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:12.979316 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 422/830] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:12 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:03:12.995320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e3955e-9524-4389-bc20-957a5a5d9430 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:13.004166 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c639c-836c-41e6-9229-5d39216956ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:13.005483 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c639c-836c-41e6-9229-5d39216956ec None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:13.005913 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c639c-836c-41e6-9229-5d39216956ec None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:13.006329 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c639c-836c-41e6-9229-5d39216956ec None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:13.007711 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812c639c-836c-41e6-9229-5d39216956ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:13.008168 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812c639c-836c-41e6-9229-5d39216956ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:13.010391 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812c639c-836c-41e6-9229-5d39216956ec None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:13.015705 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:13.033011 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 409/831] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:13.034781 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c639c-836c-41e6-9229-5d39216956ec None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:13.129968 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e3955e-9524-4389-bc20-957a5a5d9430 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:13.169269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d0b7f4-9e14-4837-a3c2-e9790f06fd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:13.170106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0b7f4-9e14-4837-a3c2-e9790f06fd80 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:13.170390 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0b7f4-9e14-4837-a3c2-e9790f06fd80 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:13.170665 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0b7f4-9e14-4837-a3c2-e9790f06fd80 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:13.171345 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d0b7f4-9e14-4837-a3c2-e9790f06fd80 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:13.172729 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d0b7f4-9e14-4837-a3c2-e9790f06fd80 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:13.172997 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c639c-836c-41e6-9229-5d39216956ec None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:13.186100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-e2d0b7f4-9e14-4837-a3c2-e9790f06fd80 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 03 16:03:13.186100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:13.186100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 16:03:13.186100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 16:03:13.186100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 16:03:13.186100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 16:03:13.186100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 16:03:13.186100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:13.186100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 03 16:03:13.186100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:13.186100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 16:03:13.186100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:13.186100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:13.186100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:13.186100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:13.186100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:13.186100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:13.187100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:13.187100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:13.187100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:13.187100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:13.187100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:13.187100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:13.187100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 16:03:13.187100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 16:03:13.187100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 16:03:13.187100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 16:03:13.187100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:13.187100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:13.187100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:13.187100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:13.187100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:13.187100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:13.187980 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:13.187980 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:13.187980 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:13.187980 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:13.187980 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:13.187980 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:13.187980 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:13.187980 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:13.187980 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:13.187980 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:13.187980 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 16:03:13.187980 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:13.187980 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 03 16:03:13.187980 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:13.193468 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 423/832] 10.222.0.41 () {56 vars in 1114 bytes} [Fri Jun 3 16:03:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:03:13.194751 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d0b7f4-9e14-4837-a3c2-e9790f06fd80 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:13.232243 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a6c03-625e-47ab-b726-3ca9ee5c50a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:13.233066 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a6c03-625e-47ab-b726-3ca9ee5c50a0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:13.233355 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a6c03-625e-47ab-b726-3ca9ee5c50a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:13.233634 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a6c03-625e-47ab-b726-3ca9ee5c50a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:13.253483 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187a6c03-625e-47ab-b726-3ca9ee5c50a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:13.254962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187a6c03-625e-47ab-b726-3ca9ee5c50a0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:13.274419 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-187a6c03-625e-47ab-b726-3ca9ee5c50a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:03:13.274419 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:13.274419 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:13.274419 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:13.274419 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:13.274419 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:13.274419 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:13.274419 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:13.274419 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:13.274419 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:13.274419 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:13.274419 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:13.274419 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:03:13.274419 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:03:13.274419 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:03:13.275383 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:03:13.275383 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:13.275383 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:13.275383 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:13.275383 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:13.275383 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:13.275383 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:13.275383 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:13.275383 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:13.275383 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:13.275383 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:13.275383 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:13.275383 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:13.275383 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:13.275383 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:13.275383 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:13.275383 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:13.276272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:03:13.276272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:03:13.276272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:03:13.276272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:03:13.276272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:03:13.276272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:03:13.276272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:03:13.276272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:13.279585 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 410/833] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:03:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:13.282148 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a6c03-625e-47ab-b726-3ca9ee5c50a0 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:13.287760 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:13.306332 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:13.326765 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d0b7f4-9e14-4837-a3c2-e9790f06fd80 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:13.362495 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c68e77-2311-4867-bbf3-d2e3d282eb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:13.363160 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c68e77-2311-4867-bbf3-d2e3d282eb69 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:13.363498 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c68e77-2311-4867-bbf3-d2e3d282eb69 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:13.363777 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c68e77-2311-4867-bbf3-d2e3d282eb69 None admin] PATH_INFO: `/v3/services` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:13.364489 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c68e77-2311-4867-bbf3-d2e3d282eb69 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:13.364797 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c68e77-2311-4867-bbf3-d2e3d282eb69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:13.366158 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c68e77-2311-4867-bbf3-d2e3d282eb69 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:13.371321 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 424/834] 10.222.0.41 () {56 vars in 1114 bytes} [Fri Jun 3 16:03:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:13.376514 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c68e77-2311-4867-bbf3-d2e3d282eb69 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:13.401329 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a6c03-625e-47ab-b726-3ca9ee5c50a0 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:13.440561 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81d217-cf5f-47bf-86fd-78d5159cc056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:13.441630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81d217-cf5f-47bf-86fd-78d5159cc056 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:13.442115 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81d217-cf5f-47bf-86fd-78d5159cc056 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:13.442634 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81d217-cf5f-47bf-86fd-78d5159cc056 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:13.443659 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c81d217-cf5f-47bf-86fd-78d5159cc056 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:13.444175 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c81d217-cf5f-47bf-86fd-78d5159cc056 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:13.445811 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c81d217-cf5f-47bf-86fd-78d5159cc056 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:13.458554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 411/835] 10.222.0.41 () {56 vars in 1148 bytes} [Fri Jun 3 16:03:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:13.467338 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81d217-cf5f-47bf-86fd-78d5159cc056 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:13.501218 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c68e77-2311-4867-bbf3-d2e3d282eb69 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:13.529837 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:13.536263 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4cf58-1ae3-4cce-9238-b0f97f74601c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:13.537017 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4cf58-1ae3-4cce-9238-b0f97f74601c None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:13.537368 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4cf58-1ae3-4cce-9238-b0f97f74601c None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:13.537677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4cf58-1ae3-4cce-9238-b0f97f74601c None admin] PATH_INFO: `/v3/services` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:13.538382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a4cf58-1ae3-4cce-9238-b0f97f74601c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:13.538689 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a4cf58-1ae3-4cce-9238-b0f97f74601c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:13.540088 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a4cf58-1ae3-4cce-9238-b0f97f74601c None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:13.544792 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 425/836] 10.222.0.41 () {56 vars in 1114 bytes} [Fri Jun 3 16:03:13 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:13.545987 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c87a8b-e206-4a81-8ca9-84f091ddfe90 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:13.546270 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c87a8b-e206-4a81-8ca9-84f091ddfe90 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:13.546554 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c87a8b-e206-4a81-8ca9-84f091ddfe90 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:13.552110 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 426/837] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:13.553524 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe601d-5434-4cf9-b0d1-306f5ffff008 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:13.553811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe601d-5434-4cf9-b0d1-306f5ffff008 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:13.554089 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe601d-5434-4cf9-b0d1-306f5ffff008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:13.575421 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-e3fe601d-5434-4cf9-b0d1-306f5ffff008 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:13.576914 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-e3fe601d-5434-4cf9-b0d1-306f5ffff008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:13.580948 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:13.584990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81d217-cf5f-47bf-86fd-78d5159cc056 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:13.625154 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2128a91-79f8-4fd4-809b-aa740c2f5766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:13.626067 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2128a91-79f8-4fd4-809b-aa740c2f5766 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:13.626356 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2128a91-79f8-4fd4-809b-aa740c2f5766 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:13.626639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2128a91-79f8-4fd4-809b-aa740c2f5766 None admin] PATH_INFO: `/v3/projects/c274254e510a468e9acbecc6298301cc/users/875140cf51d9410da45f8c3805b26204/roles/fe695aedbe8249a28c992a19264bc34b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:13.645878 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2128a91-79f8-4fd4-809b-aa740c2f5766 None admin] RBAC: Authorizing `identity:create_grant(project_id=c274254e510a468e9acbecc6298301cc, user_id=875140cf51d9410da45f8c3805b26204, role_id=fe695aedbe8249a28c992a19264bc34b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:13.647384 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2128a91-79f8-4fd4-809b-aa740c2f5766 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:13.667696 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 412/838] 10.222.0.41 () {58 vars in 1439 bytes} [Fri Jun 3 16:03:13 2022] PUT /identity/v3/projects/c274254e510a468e9acbecc6298301cc/users/875140cf51d9410da45f8c3805b26204/roles/fe695aedbe8249a28c992a19264bc34b => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:03:13.679479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 427/839] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:13 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:03:13.699198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2128a91-79f8-4fd4-809b-aa740c2f5766 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:13.825267 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2128a91-79f8-4fd4-809b-aa740c2f5766 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:13.843680 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:13.865549 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525301cf-793d-4179-a705-c88199535968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:13.866663 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525301cf-793d-4179-a705-c88199535968 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:13.867125 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525301cf-793d-4179-a705-c88199535968 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:13.867599 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525301cf-793d-4179-a705-c88199535968 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:13.879695 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525301cf-793d-4179-a705-c88199535968 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:03:13.887689 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-525301cf-793d-4179-a705-c88199535968 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:03:13.887689 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:13.887689 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:13.887689 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:13.887689 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:13.887689 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:13.887689 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:13.887689 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:13.887689 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:13.887689 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:13.887689 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:13.887689 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:13.887689 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:03:13.887689 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:03:13.887689 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:03:13.887689 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:03:13.888600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:03:13.888600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:03:13.888600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:03:13.888600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:03:13.888600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:13.888600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:13.888600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:13.888600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:13.888600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:13.888600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:13.888600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:13.888600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:13.888600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:13.888600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:13.888600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:13.888600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:13.889444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:13.889444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:13.889444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:13.889444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:13.889444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:03:13.889444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:03:13.889444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:03:13.889444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:03:13.889444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:03:13.889444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:03:13.889444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:03:13.889444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:13.891800 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 413/840] 10.222.0.41 () {56 vars in 1096 bytes} [Fri Jun 3 16:03:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:13.897803 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe601d-5434-4cf9-b0d1-306f5ffff008 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:14.004070 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe601d-5434-4cf9-b0d1-306f5ffff008 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:14.039815 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245b88c-413c-4b6e-97a9-45c6e50ea7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:14.040483 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245b88c-413c-4b6e-97a9-45c6e50ea7f8 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:14.040765 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245b88c-413c-4b6e-97a9-45c6e50ea7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:14.041040 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245b88c-413c-4b6e-97a9-45c6e50ea7f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:14.041783 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d245b88c-413c-4b6e-97a9-45c6e50ea7f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:14.042089 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d245b88c-413c-4b6e-97a9-45c6e50ea7f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:14.043481 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d245b88c-413c-4b6e-97a9-45c6e50ea7f8 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:14.048751 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:14.053457 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 428/841] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:03:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:14.058968 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525301cf-793d-4179-a705-c88199535968 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:14.168426 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525301cf-793d-4179-a705-c88199535968 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:14.205842 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50851c-8d01-4c7e-b82b-e88c4e59f026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:14.207067 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50851c-8d01-4c7e-b82b-e88c4e59f026 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:14.207666 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50851c-8d01-4c7e-b82b-e88c4e59f026 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:14.208165 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50851c-8d01-4c7e-b82b-e88c4e59f026 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:14.220197 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d50851c-8d01-4c7e-b82b-e88c4e59f026 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:14.222371 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d50851c-8d01-4c7e-b82b-e88c4e59f026 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:14.241437 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-2d50851c-8d01-4c7e-b82b-e88c4e59f026 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:14.241437 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:14.241437 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:14.241437 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:14.241437 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:14.241437 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:14.241437 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:14.241437 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:14.241437 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:14.241437 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:14.241437 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:14.241437 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:14.241437 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:03:14.241437 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:03:14.241437 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:03:14.242407 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:03:14.242407 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:14.242407 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:14.242407 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:14.242407 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:14.242407 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:14.242407 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:14.242407 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:14.242407 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:14.242407 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:14.242407 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:14.242407 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:14.242407 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:14.242407 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:14.242407 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:14.242407 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:14.242407 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:14.243366 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:03:14.243366 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:03:14.243366 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:14.243366 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:14.245197 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 414/842] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:14.250859 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245b88c-413c-4b6e-97a9-45c6e50ea7f8 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:14.365159 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245b88c-413c-4b6e-97a9-45c6e50ea7f8 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:14.407406 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84a5fe-e2b4-4a17-9a9a-423a55a69c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:14.408097 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84a5fe-e2b4-4a17-9a9a-423a55a69c9e None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:14.408426 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84a5fe-e2b4-4a17-9a9a-423a55a69c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:14.408708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84a5fe-e2b4-4a17-9a9a-423a55a69c9e None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:14.409457 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f84a5fe-e2b4-4a17-9a9a-423a55a69c9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:14.409769 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f84a5fe-e2b4-4a17-9a9a-423a55a69c9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:14.411188 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f84a5fe-e2b4-4a17-9a9a-423a55a69c9e None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:14.425536 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 429/843] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:03:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:14.430990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50851c-8d01-4c7e-b82b-e88c4e59f026 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:14.459095 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:14.542832 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50851c-8d01-4c7e-b82b-e88c4e59f026 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:14.578612 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be074b5-a357-45d1-a4d6-3efd9aecf9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:14.579431 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be074b5-a357-45d1-a4d6-3efd9aecf9fd None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:14.579722 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be074b5-a357-45d1-a4d6-3efd9aecf9fd None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:14.580003 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be074b5-a357-45d1-a4d6-3efd9aecf9fd None admin] PATH_INFO: `/v3/users/swift` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:14.587979 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be074b5-a357-45d1-a4d6-3efd9aecf9fd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:14.589425 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be074b5-a357-45d1-a4d6-3efd9aecf9fd None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:14.603964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-1be074b5-a357-45d1-a4d6-3efd9aecf9fd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 03 16:03:14.603964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:14.603964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:14.603964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:14.603964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:14.603964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:14.603964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:14.603964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:14.603964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:14.603964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:14.603964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:14.603964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:14.603964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:03:14.603964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:03:14.603964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:03:14.603964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:03:14.604935 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:14.604935 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:14.604935 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:03:14.604935 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:03:14.604935 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:03:14.604935 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:03:14.604935 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:14.604935 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:14.604935 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:14.604935 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:14.604935 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:14.604935 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:14.604935 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:14.604935 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:14.604935 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:14.604935 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:14.604935 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:14.604935 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:14.606086 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:14.606086 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:14.606086 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:03:14.606086 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:03:14.606086 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:03:14.606086 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:03:14.606086 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:03:14.606086 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:03:14.606086 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 03 16:03:14.606086 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:14.606086 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 415/844] 10.222.0.41 () {56 vars in 1096 bytes} [Fri Jun 3 16:03:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:14.614790 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84a5fe-e2b4-4a17-9a9a-423a55a69c9e None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:14.683219 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:14.727260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84a5fe-e2b4-4a17-9a9a-423a55a69c9e None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:14.746395 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ae7e1-6ed1-4860-b43d-6a50eae40ebc None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:14.746688 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ae7e1-6ed1-4860-b43d-6a50eae40ebc None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:14.746967 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ae7e1-6ed1-4860-b43d-6a50eae40ebc None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:14.748569 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 416/845] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:14.754026 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50e4f0-2719-4a20-bfa0-800990a5ff1e None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:14.754318 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50e4f0-2719-4a20-bfa0-800990a5ff1e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:14.754600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50e4f0-2719-4a20-bfa0-800990a5ff1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:14.764649 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfb346-8b79-4213-adb6-396f5e4c7020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:14.765513 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfb346-8b79-4213-adb6-396f5e4c7020 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:14.765966 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfb346-8b79-4213-adb6-396f5e4c7020 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:14.766389 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfb346-8b79-4213-adb6-396f5e4c7020 None admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:14.767462 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcfb346-8b79-4213-adb6-396f5e4c7020 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:14.767947 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcfb346-8b79-4213-adb6-396f5e4c7020 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:14.769553 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcfb346-8b79-4213-adb6-396f5e4c7020 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:14.772374 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-da50e4f0-2719-4a20-bfa0-800990a5ff1e None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:14.773769 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-da50e4f0-2719-4a20-bfa0-800990a5ff1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:14.790894 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 430/846] 10.222.0.41 () {56 vars in 1135 bytes} [Fri Jun 3 16:03:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:14.796502 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfb346-8b79-4213-adb6-396f5e4c7020 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:14.848359 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 417/847] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:14 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 16:03:14.870511 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50e4f0-2719-4a20-bfa0-800990a5ff1e None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:14.886372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:14.900917 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfb346-8b79-4213-adb6-396f5e4c7020 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:14.933039 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:14.939276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ec8e4-4a62-4346-9660-972fdc97d814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:14.940956 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ec8e4-4a62-4346-9660-972fdc97d814 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:14.941242 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ec8e4-4a62-4346-9660-972fdc97d814 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:14.941521 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ec8e4-4a62-4346-9660-972fdc97d814 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:14.992449 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50e4f0-2719-4a20-bfa0-800990a5ff1e None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:15.029705 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c45bed3-d729-406f-b81d-11041c7ea29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:15.030516 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c45bed3-d729-406f-b81d-11041c7ea29c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:15.030795 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c45bed3-d729-406f-b81d-11041c7ea29c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:15.031068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c45bed3-d729-406f-b81d-11041c7ea29c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:15.031757 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c45bed3-d729-406f-b81d-11041c7ea29c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:15.033141 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c45bed3-d729-406f-b81d-11041c7ea29c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:15.033281 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ec8e4-4a62-4346-9660-972fdc97d814 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:15.036349 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ec8e4-4a62-4346-9660-972fdc97d814 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:15.045990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-7c45bed3-d729-406f-b81d-11041c7ea29c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 03 16:03:15.045990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:15.045990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 16:03:15.045990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 16:03:15.045990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 16:03:15.045990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 16:03:15.045990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 16:03:15.045990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:15.045990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 03 16:03:15.045990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:15.045990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 16:03:15.045990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:15.045990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:15.045990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:15.045990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:15.045990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:15.046929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:15.046929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:15.046929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:15.046929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:15.046929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:15.046929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:15.046929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:15.046929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 16:03:15.046929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 16:03:15.046929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 16:03:15.046929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 16:03:15.046929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:15.046929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:15.046929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:15.046929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:15.046929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:15.047792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:15.047792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:15.047792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:15.047792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:15.047792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:15.047792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:15.047792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:15.047792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:15.047792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:15.047792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:15.047792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:15.047792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 16:03:15.047792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:15.047792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 03 16:03:15.047792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:15.047792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 418/848] 10.222.0.41 () {56 vars in 1132 bytes} [Fri Jun 3 16:03:14 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:03:15.052969 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c45bed3-d729-406f-b81d-11041c7ea29c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:15.151081 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:15.162040 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 431/849] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:03:14 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 03 16:03:15.169230 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ec8e4-4a62-4346-9660-972fdc97d814 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:15.182061 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c45bed3-d729-406f-b81d-11041c7ea29c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:15.192233 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:15.219701 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d91a4-54b5-4031-998d-952ffbba8e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:15.220544 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d91a4-54b5-4031-998d-952ffbba8e27 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:15.221512 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d91a4-54b5-4031-998d-952ffbba8e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:15.221958 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d91a4-54b5-4031-998d-952ffbba8e27 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:15.222760 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762d91a4-54b5-4031-998d-952ffbba8e27 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:15.223274 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762d91a4-54b5-4031-998d-952ffbba8e27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:15.226647 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762d91a4-54b5-4031-998d-952ffbba8e27 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:15.237364 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 419/850] 10.222.0.41 () {56 vars in 1126 bytes} [Fri Jun 3 16:03:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:15.237364 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d91a4-54b5-4031-998d-952ffbba8e27 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:15.302541 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ec8e4-4a62-4346-9660-972fdc97d814 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:15.336639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d91a4-54b5-4031-998d-952ffbba8e27 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:15.372070 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61585437-87b5-42f6-a023-75fd6df56488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:15.372897 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61585437-87b5-42f6-a023-75fd6df56488 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:15.373769 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61585437-87b5-42f6-a023-75fd6df56488 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:15.374085 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61585437-87b5-42f6-a023-75fd6df56488 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:15.391969 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61585437-87b5-42f6-a023-75fd6df56488 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:15.393937 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61585437-87b5-42f6-a023-75fd6df56488 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:15.395959 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11530c2-91c1-4085-ab26-731fa38af503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:15.396738 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11530c2-91c1-4085-ab26-731fa38af503 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:15.397032 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11530c2-91c1-4085-ab26-731fa38af503 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:15.397419 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11530c2-91c1-4085-ab26-731fa38af503 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:15.398182 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11530c2-91c1-4085-ab26-731fa38af503 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:15.398581 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11530c2-91c1-4085-ab26-731fa38af503 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:15.400887 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11530c2-91c1-4085-ab26-731fa38af503 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:15.408014 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 420/851] 10.222.0.41 () {56 vars in 1126 bytes} [Fri Jun 3 16:03:15 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:15.425079 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-61585437-87b5-42f6-a023-75fd6df56488 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:15.425079 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:15.425079 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:15.425079 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:15.425079 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:15.425079 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:15.425079 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:15.425079 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:15.425079 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:15.425079 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:15.425079 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:15.425079 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:15.425079 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:03:15.425079 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:03:15.425079 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:03:15.426101 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:03:15.426101 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:15.426101 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:15.426101 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:15.426101 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:15.426101 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:15.426101 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:15.426101 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:15.426101 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:15.426101 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:15.426101 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:15.426101 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:15.426101 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:15.426101 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:15.426101 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:15.426101 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:15.426101 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:15.426979 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:03:15.426979 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:03:15.426979 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:15.426979 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:15.426979 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 432/852] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:15.432310 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11530c2-91c1-4085-ab26-731fa38af503 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:15.443122 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:15.549199 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11530c2-91c1-4085-ab26-731fa38af503 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:15.587627 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d6d53-e058-4580-a43b-e20324da35ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:15.588630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d6d53-e058-4580-a43b-e20324da35ad None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:15.589138 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d6d53-e058-4580-a43b-e20324da35ad None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:15.589597 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d6d53-e058-4580-a43b-e20324da35ad None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:15.590527 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976d6d53-e058-4580-a43b-e20324da35ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:15.591030 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976d6d53-e058-4580-a43b-e20324da35ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:15.592647 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976d6d53-e058-4580-a43b-e20324da35ad None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:15.607816 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 421/853] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:15.613486 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61585437-87b5-42f6-a023-75fd6df56488 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:15.693388 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:15.727657 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61585437-87b5-42f6-a023-75fd6df56488 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:15.764108 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102ed55-a8c9-4887-8beb-f6400ab6a294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:15.764912 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102ed55-a8c9-4887-8beb-f6400ab6a294 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:15.765194 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102ed55-a8c9-4887-8beb-f6400ab6a294 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:15.765471 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102ed55-a8c9-4887-8beb-f6400ab6a294 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:15.772761 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102ed55-a8c9-4887-8beb-f6400ab6a294 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:15.774217 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102ed55-a8c9-4887-8beb-f6400ab6a294 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:15.788884 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-a102ed55-a8c9-4887-8beb-f6400ab6a294 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:03:15.788884 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:15.788884 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:15.788884 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:15.788884 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:15.788884 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:15.788884 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:15.788884 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:15.788884 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:15.788884 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:15.788884 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:15.788884 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:15.788884 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:03:15.788884 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:03:15.788884 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:03:15.789779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:03:15.789779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:15.789779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:15.789779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:15.789779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:15.789779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:15.789779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:15.789779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:15.789779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:15.789779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:15.789779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:15.789779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:15.789779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:15.789779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:15.789779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:15.789779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:15.789779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:15.790657 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:03:15.790657 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:03:15.790657 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:03:15.790657 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:03:15.790657 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:03:15.790657 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:03:15.790657 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:03:15.790657 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:15.790657 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 433/854] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:03:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:15.796194 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d6d53-e058-4580-a43b-e20324da35ad None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:15.891313 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:15.909135 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d6d53-e058-4580-a43b-e20324da35ad None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:15.944806 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a3bfc6-0230-4a1c-adaf-ca94c0c9f708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:15.945802 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a3bfc6-0230-4a1c-adaf-ca94c0c9f708 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:15.946310 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a3bfc6-0230-4a1c-adaf-ca94c0c9f708 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:15.946820 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a3bfc6-0230-4a1c-adaf-ca94c0c9f708 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:15.947837 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a3bfc6-0230-4a1c-adaf-ca94c0c9f708 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:15.948353 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a3bfc6-0230-4a1c-adaf-ca94c0c9f708 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:15.950115 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a3bfc6-0230-4a1c-adaf-ca94c0c9f708 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:15.960640 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c64d6-9112-4561-a5e1-e1aa4f8be927 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:15.960925 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c64d6-9112-4561-a5e1-e1aa4f8be927 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:15.961215 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c64d6-9112-4561-a5e1-e1aa4f8be927 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:15.962834 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 434/855] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 16:03:15.964410 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 422/856] 10.222.0.41 () {56 vars in 1148 bytes} [Fri Jun 3 16:03:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:15.971436 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d465a-3996-463b-985a-933b60aa0311 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:15.971436 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d465a-3996-463b-985a-933b60aa0311 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:15.971436 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d465a-3996-463b-985a-933b60aa0311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:15.972583 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a3bfc6-0230-4a1c-adaf-ca94c0c9f708 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:15.987653 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-c68d465a-3996-463b-985a-933b60aa0311 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:15.988865 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-c68d465a-3996-463b-985a-933b60aa0311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:16.071034 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 435/857] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:15 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:03:16.087689 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a3bfc6-0230-4a1c-adaf-ca94c0c9f708 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:16.093175 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d465a-3996-463b-985a-933b60aa0311 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:16.115136 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:16.125012 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe6c44-b487-48e0-bd40-45e5607ae072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:16.125680 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe6c44-b487-48e0-bd40-45e5607ae072 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:16.126009 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe6c44-b487-48e0-bd40-45e5607ae072 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:16.126299 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe6c44-b487-48e0-bd40-45e5607ae072 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:16.127781 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe6c44-b487-48e0-bd40-45e5607ae072 None admin] RBAC: Adding query filter params (role.id=75936a608b8149099ff54e317f3a6740, scope.project.id=c274254e510a468e9acbecc6298301cc, user.id=6c5f513b781a4a73b443cded13359eda) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:16.128079 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe6c44-b487-48e0-bd40-45e5607ae072 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:16.129533 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe6c44-b487-48e0-bd40-45e5607ae072 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:16.140470 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 423/858] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:03:15 2022] GET /identity/v3/role_assignments?user.id=6c5f513b781a4a73b443cded13359eda&scope.project.id=c274254e510a468e9acbecc6298301cc&role.id=75936a608b8149099ff54e317f3a6740 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:16.163728 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:16.178210 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52520bad-83ba-4f17-91e9-36d71786bf09 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:16.178210 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52520bad-83ba-4f17-91e9-36d71786bf09 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:16.178210 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52520bad-83ba-4f17-91e9-36d71786bf09 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:16.178210 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 424/859] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 16:03:16.186046 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737004d-632c-481e-bd96-50a730dcc6d1 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:16.186680 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737004d-632c-481e-bd96-50a730dcc6d1 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:16.187188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737004d-632c-481e-bd96-50a730dcc6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:16.207721 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-6737004d-632c-481e-bd96-50a730dcc6d1 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:16.209511 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6737004d-632c-481e-bd96-50a730dcc6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:16.219810 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d465a-3996-463b-985a-933b60aa0311 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:16.258883 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c198b641-1027-4aa9-ae33-6bac635c9ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:16.259591 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c198b641-1027-4aa9-ae33-6bac635c9ec8 None admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:16.259874 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c198b641-1027-4aa9-ae33-6bac635c9ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:16.260149 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c198b641-1027-4aa9-ae33-6bac635c9ec8 None admin] PATH_INFO: `/v3/services` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:16.260880 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c198b641-1027-4aa9-ae33-6bac635c9ec8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:16.262623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c198b641-1027-4aa9-ae33-6bac635c9ec8 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:16.283560 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 436/860] 10.222.0.41 () {60 vars in 1141 bytes} [Fri Jun 3 16:03:16 2022] POST /identity/v3/services => generated 251 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:03:16.303493 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 425/861] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:16 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 16:03:16.329132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c198b641-1027-4aa9-ae33-6bac635c9ec8 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:16.445272 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c198b641-1027-4aa9-ae33-6bac635c9ec8 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:16.466967 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:16.486397 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db2d2d-8341-4667-8d20-0d7b474398d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:16.487739 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db2d2d-8341-4667-8d20-0d7b474398d3 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:16.488027 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db2d2d-8341-4667-8d20-0d7b474398d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:16.488304 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db2d2d-8341-4667-8d20-0d7b474398d3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:16.498953 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98db2d2d-8341-4667-8d20-0d7b474398d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:03:16.506750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-98db2d2d-8341-4667-8d20-0d7b474398d3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:03:16.506750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:16.506750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:16.506750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:16.506750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:16.506750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:16.506750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:16.506750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:16.506750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:16.506750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:16.506750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:16.506750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:16.506750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:03:16.506750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:03:16.506750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:03:16.506750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:03:16.507728 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:03:16.507728 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:03:16.507728 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:03:16.507728 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:03:16.507728 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:16.507728 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:16.507728 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:16.507728 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:16.507728 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:16.507728 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:16.507728 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:16.507728 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:16.507728 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:16.507728 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:16.507728 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:16.507728 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:16.508590 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:16.508590 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:16.508590 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:16.508590 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:16.508590 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:03:16.508590 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:03:16.508590 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:03:16.508590 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:03:16.508590 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:03:16.508590 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:03:16.508590 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:03:16.508590 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:16.508590 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 437/862] 10.222.0.41 () {56 vars in 1102 bytes} [Fri Jun 3 16:03:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:16.513992 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737004d-632c-481e-bd96-50a730dcc6d1 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:16.619824 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737004d-632c-481e-bd96-50a730dcc6d1 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:16.638263 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:16.659627 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099b2fc-5b61-42c7-969c-1c94291edc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:16.660310 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099b2fc-5b61-42c7-969c-1c94291edc52 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:16.660600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099b2fc-5b61-42c7-969c-1c94291edc52 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:16.660884 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099b2fc-5b61-42c7-969c-1c94291edc52 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:16.661645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f099b2fc-5b61-42c7-969c-1c94291edc52 None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:16.661958 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f099b2fc-5b61-42c7-969c-1c94291edc52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:16.663367 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f099b2fc-5b61-42c7-969c-1c94291edc52 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:16.672465 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 426/863] 10.222.0.41 () {56 vars in 1103 bytes} [Fri Jun 3 16:03:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:16.678052 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db2d2d-8341-4667-8d20-0d7b474398d3 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:16.787706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db2d2d-8341-4667-8d20-0d7b474398d3 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:16.823082 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4648cf9-1256-47a0-beb3-334892e85b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:16.823920 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4648cf9-1256-47a0-beb3-334892e85b8a None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:16.824209 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4648cf9-1256-47a0-beb3-334892e85b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:16.824492 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4648cf9-1256-47a0-beb3-334892e85b8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:16.835598 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4648cf9-1256-47a0-beb3-334892e85b8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:16.837112 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4648cf9-1256-47a0-beb3-334892e85b8a None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:16.854623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-d4648cf9-1256-47a0-beb3-334892e85b8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:16.854623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:16.854623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:16.854623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:16.854623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:16.854623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:16.854623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:16.854623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:16.854623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:16.854623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:16.854623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:16.854623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:16.854623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:03:16.854623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:03:16.854623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:03:16.855620 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:03:16.855620 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:16.855620 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:16.855620 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:16.855620 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:16.855620 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:16.855620 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:16.855620 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:16.855620 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:16.855620 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:16.855620 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:16.855620 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:16.855620 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:16.855620 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:16.855620 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:16.855620 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:16.855620 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:16.857508 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:03:16.857508 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:03:16.857508 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:16.857508 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:16.857508 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 438/864] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:16.862184 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099b2fc-5b61-42c7-969c-1c94291edc52 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:16.971354 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099b2fc-5b61-42c7-969c-1c94291edc52 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:17.007617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e9344-94f1-4026-9e10-21cffbd1a600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:17.008286 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e9344-94f1-4026-9e10-21cffbd1a600 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:17.008700 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e9344-94f1-4026-9e10-21cffbd1a600 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:17.008984 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e9344-94f1-4026-9e10-21cffbd1a600 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:17.009787 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74e9344-94f1-4026-9e10-21cffbd1a600 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:17.010096 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74e9344-94f1-4026-9e10-21cffbd1a600 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:17.011500 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74e9344-94f1-4026-9e10-21cffbd1a600 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:17.024457 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 427/865] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:17.030215 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4648cf9-1256-47a0-beb3-334892e85b8a None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:17.063684 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:17.135424 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4648cf9-1256-47a0-beb3-334892e85b8a None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:17.171897 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c21527-e586-4187-b9cc-c6518f3d3462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:17.172719 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c21527-e586-4187-b9cc-c6518f3d3462 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:17.173011 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c21527-e586-4187-b9cc-c6518f3d3462 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:17.173289 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c21527-e586-4187-b9cc-c6518f3d3462 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:17.181368 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c21527-e586-4187-b9cc-c6518f3d3462 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:17.182804 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c21527-e586-4187-b9cc-c6518f3d3462 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:17.198153 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-75c21527-e586-4187-b9cc-c6518f3d3462 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 03 16:03:17.198153 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:17.198153 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:17.198153 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:17.198153 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:17.198153 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:17.198153 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:17.198153 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:17.198153 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:17.198153 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:17.198153 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:17.198153 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:17.198153 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:03:17.198153 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:03:17.198153 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:03:17.198153 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:03:17.199191 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:17.199191 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:17.199191 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:03:17.199191 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:03:17.199191 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:03:17.199191 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:03:17.199191 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:17.199191 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:17.199191 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:17.199191 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:17.199191 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:17.199191 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:17.199191 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:17.199191 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:17.199191 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:17.199191 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:17.199191 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:17.199191 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:17.200244 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:17.200244 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:17.200244 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:03:17.200244 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:03:17.200244 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:03:17.200244 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:03:17.200244 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:03:17.200244 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:03:17.200244 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 03 16:03:17.200244 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:17.200244 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 439/866] 10.222.0.41 () {56 vars in 1117 bytes} [Fri Jun 3 16:03:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:17.208360 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e9344-94f1-4026-9e10-21cffbd1a600 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:17.263038 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:17.337400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e9344-94f1-4026-9e10-21cffbd1a600 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:17.372996 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d487c5-2d7f-41ae-8e85-ec523a05bc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:17.373931 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d487c5-2d7f-41ae-8e85-ec523a05bc4b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:17.374378 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d487c5-2d7f-41ae-8e85-ec523a05bc4b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:17.374814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d487c5-2d7f-41ae-8e85-ec523a05bc4b None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:17.375844 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d487c5-2d7f-41ae-8e85-ec523a05bc4b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:17.376330 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d487c5-2d7f-41ae-8e85-ec523a05bc4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:17.377958 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d487c5-2d7f-41ae-8e85-ec523a05bc4b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:17.395981 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 428/867] 10.222.0.41 () {56 vars in 1149 bytes} [Fri Jun 3 16:03:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:17.401661 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c21527-e586-4187-b9cc-c6518f3d3462 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:17.507436 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:17.516448 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c21527-e586-4187-b9cc-c6518f3d3462 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:17.552337 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e3acfe-5232-4484-a57e-f151384e416c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:17.552990 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3acfe-5232-4484-a57e-f151384e416c None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:17.553279 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3acfe-5232-4484-a57e-f151384e416c None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:17.553549 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3acfe-5232-4484-a57e-f151384e416c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:17.623217 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e3acfe-5232-4484-a57e-f151384e416c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:17.624609 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e3acfe-5232-4484-a57e-f151384e416c None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:17.714993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:17.724501 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 440/868] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:03:17 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 16:03:17.729947 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d487c5-2d7f-41ae-8e85-ec523a05bc4b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:17.804995 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122de41-3928-47c3-aa2e-0941f4fc2117 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:17.805354 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122de41-3928-47c3-aa2e-0941f4fc2117 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:17.805556 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122de41-3928-47c3-aa2e-0941f4fc2117 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:17.807114 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 441/869] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:17.812302 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cb310-c1b0-4894-915d-9975c2d58f80 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:17.812439 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cb310-c1b0-4894-915d-9975c2d58f80 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:17.812728 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cb310-c1b0-4894-915d-9975c2d58f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:17.831103 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-4b1cb310-c1b0-4894-915d-9975c2d58f80 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:17.832671 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:17.834238 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-4b1cb310-c1b0-4894-915d-9975c2d58f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:17.840101 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d487c5-2d7f-41ae-8e85-ec523a05bc4b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:17.884833 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c15d3-6af9-49b3-ab3f-61153c4772d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:17.885664 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c15d3-6af9-49b3-ab3f-61153c4772d2 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:17.886016 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c15d3-6af9-49b3-ab3f-61153c4772d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:17.886849 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c15d3-6af9-49b3-ab3f-61153c4772d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:17.900973 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c15d3-6af9-49b3-ab3f-61153c4772d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:17.902988 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c15d3-6af9-49b3-ab3f-61153c4772d2 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:17.923541 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 442/870] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:17 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 16:03:17.924659 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-772c15d3-6af9-49b3-ab3f-61153c4772d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:17.924659 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:17.924659 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:17.924659 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:17.924659 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:17.924659 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:17.924659 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:17.924659 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:17.924659 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:17.924659 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:17.924659 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:17.924659 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:17.924659 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:03:17.924659 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:03:17.924659 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:03:17.925547 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:03:17.925547 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:17.925547 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:17.925547 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:17.925547 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:17.925547 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:17.925547 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:17.925547 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:17.925547 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:17.925547 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:17.925547 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:17.925547 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:17.925547 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:17.925547 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:17.925547 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:17.925547 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:17.925547 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:17.926456 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:03:17.926456 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:03:17.926456 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:17.926456 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:17.926456 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 429/871] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:17.932293 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1cb310-c1b0-4894-915d-9975c2d58f80 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:17.943585 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c15d3-6af9-49b3-ab3f-61153c4772d2 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:18.051004 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c15d3-6af9-49b3-ab3f-61153c4772d2 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:18.061372 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1cb310-c1b0-4894-915d-9975c2d58f80 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:18.089238 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b9112-d34c-4bc6-9a33-1c708bc88850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:18.090039 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b9112-d34c-4bc6-9a33-1c708bc88850 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:18.090417 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b9112-d34c-4bc6-9a33-1c708bc88850 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:18.090773 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b9112-d34c-4bc6-9a33-1c708bc88850 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:18.091700 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422b9112-d34c-4bc6-9a33-1c708bc88850 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:18.092941 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:18.093305 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422b9112-d34c-4bc6-9a33-1c708bc88850 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:18.111619 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9540b36-623c-4c32-8b55-803ab967a705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:18.112465 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9540b36-623c-4c32-8b55-803ab967a705 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:18.112761 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9540b36-623c-4c32-8b55-803ab967a705 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:18.113046 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9540b36-623c-4c32-8b55-803ab967a705 None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:18.113880 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9540b36-623c-4c32-8b55-803ab967a705 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:18.114199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9540b36-623c-4c32-8b55-803ab967a705 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:18.115966 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9540b36-623c-4c32-8b55-803ab967a705 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:18.119539 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 430/872] 10.222.0.41 () {60 vars in 1141 bytes} [Fri Jun 3 16:03:17 2022] POST /identity/v3/services => generated 269 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:03:18.132377 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 443/873] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:03:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:18.137655 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b9112-d34c-4bc6-9a33-1c708bc88850 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:18.163728 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:18.266026 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b9112-d34c-4bc6-9a33-1c708bc88850 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:18.311558 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff846f-93e9-4be9-9e37-2ddbb4f5da83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:18.312392 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff846f-93e9-4be9-9e37-2ddbb4f5da83 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:18.312679 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff846f-93e9-4be9-9e37-2ddbb4f5da83 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:18.312957 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff846f-93e9-4be9-9e37-2ddbb4f5da83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:18.321973 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ff846f-93e9-4be9-9e37-2ddbb4f5da83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:18.323515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ff846f-93e9-4be9-9e37-2ddbb4f5da83 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:18.340313 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-d8ff846f-93e9-4be9-9e37-2ddbb4f5da83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:03:18.340313 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:18.340313 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:18.340313 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:18.340313 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:18.340313 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:18.340313 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:18.340313 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:18.340313 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:18.340313 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:18.340313 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:18.340313 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:18.340313 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:03:18.340313 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:03:18.340313 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:03:18.341292 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:03:18.341292 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:18.341292 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:18.341292 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:18.341292 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:18.341292 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:18.341292 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:18.341292 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:18.341292 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:18.341292 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:18.341292 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:18.341292 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:18.341292 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:18.341292 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:18.341292 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:18.341292 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:18.341292 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:18.342252 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:03:18.342252 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:03:18.342252 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:03:18.342252 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:03:18.342252 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:03:18.342252 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:03:18.342252 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:03:18.342252 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:18.342252 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 431/874] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:03:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:18.348564 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9540b36-623c-4c32-8b55-803ab967a705 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:18.459508 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9540b36-623c-4c32-8b55-803ab967a705 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:18.495432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ec5b1-1b47-40f8-bb45-80325a9e329a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:18.496111 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ec5b1-1b47-40f8-bb45-80325a9e329a None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:18.496396 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ec5b1-1b47-40f8-bb45-80325a9e329a None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:18.496675 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ec5b1-1b47-40f8-bb45-80325a9e329a None admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:18.497471 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ec5b1-1b47-40f8-bb45-80325a9e329a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:18.497784 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ec5b1-1b47-40f8-bb45-80325a9e329a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:18.499229 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ec5b1-1b47-40f8-bb45-80325a9e329a None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:18.512567 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 444/875] 10.222.0.41 () {56 vars in 1148 bytes} [Fri Jun 3 16:03:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:18.519061 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff846f-93e9-4be9-9e37-2ddbb4f5da83 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:18.586566 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:18.621246 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7e1d4-53fd-4d16-bec2-28b4235fca35 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:18.621559 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7e1d4-53fd-4d16-bec2-28b4235fca35 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:18.621818 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7e1d4-53fd-4d16-bec2-28b4235fca35 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:18.624079 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 445/876] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:18.633302 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5bcf7-2f05-46b7-9d06-3d1e1e227253 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:18.633830 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5bcf7-2f05-46b7-9d06-3d1e1e227253 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:18.634303 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5bcf7-2f05-46b7-9d06-3d1e1e227253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:18.635228 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff846f-93e9-4be9-9e37-2ddbb4f5da83 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:18.651382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-3fd5bcf7-2f05-46b7-9d06-3d1e1e227253 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:18.652908 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-3fd5bcf7-2f05-46b7-9d06-3d1e1e227253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:18.679061 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff61b8-84e6-4415-bea8-262b85e29363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:18.679821 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff61b8-84e6-4415-bea8-262b85e29363 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:18.680104 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff61b8-84e6-4415-bea8-262b85e29363 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:18.680386 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff61b8-84e6-4415-bea8-262b85e29363 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:18.681225 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcff61b8-84e6-4415-bea8-262b85e29363 None admin] RBAC: Adding query filter params (role.id=fe695aedbe8249a28c992a19264bc34b, scope.project.id=c274254e510a468e9acbecc6298301cc, user.id=875140cf51d9410da45f8c3805b26204) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:18.681541 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcff61b8-84e6-4415-bea8-262b85e29363 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:18.682970 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcff61b8-84e6-4415-bea8-262b85e29363 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:18.689787 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 432/877] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:03:18 2022] GET /identity/v3/role_assignments?user.id=875140cf51d9410da45f8c3805b26204&scope.project.id=c274254e510a468e9acbecc6298301cc&role.id=fe695aedbe8249a28c992a19264bc34b => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:18.691027 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f3f12-e851-401f-a969-e4e6c2ef9372 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:18.691333 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f3f12-e851-401f-a969-e4e6c2ef9372 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:18.691616 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f3f12-e851-401f-a969-e4e6c2ef9372 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:18.697319 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:18.697948 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 433/878] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:18.699383 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dcb561-b240-4f2e-8464-b829c32327cf None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:18.699657 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dcb561-b240-4f2e-8464-b829c32327cf None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:18.699932 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dcb561-b240-4f2e-8464-b829c32327cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:18.718678 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-13dcb561-b240-4f2e-8464-b829c32327cf None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:18.719847 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-13dcb561-b240-4f2e-8464-b829c32327cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:18.738244 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 446/879] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:18 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 16:03:18.763500 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5bcf7-2f05-46b7-9d06-3d1e1e227253 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:18.815375 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 434/880] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:18 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:03:18.840483 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dcb561-b240-4f2e-8464-b829c32327cf None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:18.886125 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5bcf7-2f05-46b7-9d06-3d1e1e227253 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:18.910005 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:18.925325 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63301b18-6a81-47af-8fd4-4fc7cf5c14e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:18.927146 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63301b18-6a81-47af-8fd4-4fc7cf5c14e0 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:18.927626 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63301b18-6a81-47af-8fd4-4fc7cf5c14e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:18.928057 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63301b18-6a81-47af-8fd4-4fc7cf5c14e0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:18.928861 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63301b18-6a81-47af-8fd4-4fc7cf5c14e0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:18.930407 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63301b18-6a81-47af-8fd4-4fc7cf5c14e0 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:18.945993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-63301b18-6a81-47af-8fd4-4fc7cf5c14e0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 03 16:03:18.945993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:18.945993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 16:03:18.945993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 16:03:18.945993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 16:03:18.945993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 16:03:18.945993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 16:03:18.945993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:18.945993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 03 16:03:18.945993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:18.945993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 16:03:18.945993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:18.945993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:18.945993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:18.945993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:18.945993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:18.945993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:18.947053 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:18.947053 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:18.947053 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:18.947053 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:18.947053 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:18.947053 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:18.947053 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 16:03:18.947053 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 16:03:18.947053 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 16:03:18.947053 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 16:03:18.947053 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:18.947053 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:18.947053 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:18.947053 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:18.947053 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:18.947053 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:18.947938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:18.947938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:18.947938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:18.947938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:18.947938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:18.947938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:18.947938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:18.947938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:18.947938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:18.947938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:18.947938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 16:03:18.947938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:18.947938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 03 16:03:18.947938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:18.951383 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 447/881] 10.222.0.41 () {56 vars in 1114 bytes} [Fri Jun 3 16:03:18 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:03:18.959402 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63301b18-6a81-47af-8fd4-4fc7cf5c14e0 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:18.973503 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dcb561-b240-4f2e-8464-b829c32327cf None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:19.011096 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:19.016221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f050fb6-b42d-4511-b36b-8437a0c425ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:19.016413 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f050fb6-b42d-4511-b36b-8437a0c425ce None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:19.017001 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f050fb6-b42d-4511-b36b-8437a0c425ce None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:19.017001 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f050fb6-b42d-4511-b36b-8437a0c425ce None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:19.018032 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f050fb6-b42d-4511-b36b-8437a0c425ce None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:19.020242 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f050fb6-b42d-4511-b36b-8437a0c425ce None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:19.033633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-8f050fb6-b42d-4511-b36b-8437a0c425ce None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 16:03:19.033633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:19.033633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 16:03:19.033633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 16:03:19.033633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 16:03:19.033633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 16:03:19.033633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 16:03:19.033633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:19.033633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 16:03:19.033633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:19.033633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 16:03:19.033633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:19.033633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:19.033633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:19.033633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:19.033633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:19.033633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:19.034663 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:19.034663 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:19.034663 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:19.034663 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:19.034663 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:19.034663 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:19.034663 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 16:03:19.034663 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 16:03:19.034663 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 16:03:19.034663 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 16:03:19.034663 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:19.034663 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:19.034663 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:19.034663 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:19.034663 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:19.034663 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:19.035517 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:19.035517 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:19.035517 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:19.035517 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:19.035517 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:19.035517 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:19.035517 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:19.035517 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:19.035517 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:19.035517 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:19.035517 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 16:03:19.035517 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:19.035517 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 16:03:19.035517 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:19.035517 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 435/882] 10.222.0.41 () {56 vars in 1126 bytes} [Fri Jun 3 16:03:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:03:19.041252 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f050fb6-b42d-4511-b36b-8437a0c425ce None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:19.094846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63301b18-6a81-47af-8fd4-4fc7cf5c14e0 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:19.140410 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64bd621-b04f-4505-af8b-d77285229f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:19.141100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64bd621-b04f-4505-af8b-d77285229f33 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:19.141387 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64bd621-b04f-4505-af8b-d77285229f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:19.141729 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64bd621-b04f-4505-af8b-d77285229f33 None admin] PATH_INFO: `/v3/services` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:19.143512 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64bd621-b04f-4505-af8b-d77285229f33 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:19.143854 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64bd621-b04f-4505-af8b-d77285229f33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:19.146030 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64bd621-b04f-4505-af8b-d77285229f33 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:19.152977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 448/883] 10.222.0.41 () {56 vars in 1114 bytes} [Fri Jun 3 16:03:18 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:19.158815 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64bd621-b04f-4505-af8b-d77285229f33 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:19.171770 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f050fb6-b42d-4511-b36b-8437a0c425ce None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:19.197651 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:19.211559 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d8547-6d4e-4dc3-947d-798320efd33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:19.212234 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d8547-6d4e-4dc3-947d-798320efd33e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:19.212529 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d8547-6d4e-4dc3-947d-798320efd33e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:19.212827 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d8547-6d4e-4dc3-947d-798320efd33e None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:19.213740 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d8547-6d4e-4dc3-947d-798320efd33e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:19.213923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d8547-6d4e-4dc3-947d-798320efd33e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:19.215585 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d8547-6d4e-4dc3-947d-798320efd33e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:19.225151 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 436/884] 10.222.0.41 () {56 vars in 1122 bytes} [Fri Jun 3 16:03:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:19.227270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d8547-6d4e-4dc3-947d-798320efd33e None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:19.287384 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:19.292872 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64bd621-b04f-4505-af8b-d77285229f33 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:19.342829 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b02a3c9-d379-4356-9581-5d98d12586b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:19.343539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b02a3c9-d379-4356-9581-5d98d12586b9 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:19.343840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b02a3c9-d379-4356-9581-5d98d12586b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:19.344112 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b02a3c9-d379-4356-9581-5d98d12586b9 None admin] PATH_INFO: `/v3/services` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:19.344832 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b02a3c9-d379-4356-9581-5d98d12586b9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:19.345142 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b02a3c9-d379-4356-9581-5d98d12586b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:19.347176 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b02a3c9-d379-4356-9581-5d98d12586b9 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:19.353243 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 449/885] 10.222.0.41 () {56 vars in 1114 bytes} [Fri Jun 3 16:03:19 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:19.359160 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b02a3c9-d379-4356-9581-5d98d12586b9 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:19.386154 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d8547-6d4e-4dc3-947d-798320efd33e None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:19.430027 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5229b-11f6-470b-b269-666dbed80261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:19.430846 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5229b-11f6-470b-b269-666dbed80261 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:19.431128 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5229b-11f6-470b-b269-666dbed80261 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:19.431445 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5229b-11f6-470b-b269-666dbed80261 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:19.432649 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb5229b-11f6-470b-b269-666dbed80261 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:19.433104 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb5229b-11f6-470b-b269-666dbed80261 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:19.434754 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb5229b-11f6-470b-b269-666dbed80261 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:19.444701 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 437/886] 10.222.0.41 () {56 vars in 1122 bytes} [Fri Jun 3 16:03:19 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:19.472871 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b02a3c9-d379-4356-9581-5d98d12586b9 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:19.513441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ec70d-1840-47ad-a6fc-cf3e0301181c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:19.514062 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ec70d-1840-47ad-a6fc-cf3e0301181c None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:19.514303 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ec70d-1840-47ad-a6fc-cf3e0301181c None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:19.514580 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ec70d-1840-47ad-a6fc-cf3e0301181c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:19.515415 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147ec70d-1840-47ad-a6fc-cf3e0301181c None admin] RBAC: Adding query filter params (interface=public, service_id=91dc1fb68888465abf7cdac8d5df88ec, region_id=RegionOne) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:19.515731 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147ec70d-1840-47ad-a6fc-cf3e0301181c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:19.517132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147ec70d-1840-47ad-a6fc-cf3e0301181c None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:19.523826 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 450/887] 10.222.0.41 () {56 vars in 1251 bytes} [Fri Jun 3 16:03:19 2022] GET /identity/v3/endpoints?service_id=91dc1fb68888465abf7cdac8d5df88ec&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:19.529664 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5229b-11f6-470b-b269-666dbed80261 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:19.586449 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:19.646278 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5229b-11f6-470b-b269-666dbed80261 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:19.683061 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cafd04-6064-4401-a0e3-4ab64eceb9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:19.683762 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cafd04-6064-4401-a0e3-4ab64eceb9df None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:19.684049 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cafd04-6064-4401-a0e3-4ab64eceb9df None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:19.684334 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cafd04-6064-4401-a0e3-4ab64eceb9df None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:19.685025 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cafd04-6064-4401-a0e3-4ab64eceb9df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:19.686403 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cafd04-6064-4401-a0e3-4ab64eceb9df None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:19.692632 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 438/888] 10.222.0.41 () {56 vars in 1087 bytes} [Fri Jun 3 16:03:19 2022] GET /identity/v3/services => generated 1268 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:03:20.477826 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b553aa1-5712-4a13-9543-11b87c113400 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:20.478354 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b553aa1-5712-4a13-9543-11b87c113400 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:20.478802 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b553aa1-5712-4a13-9543-11b87c113400 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:20.480575 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 451/889] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:20.486141 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d404efb-cd3f-4753-8704-89ef60b7c0da None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:20.486930 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d404efb-cd3f-4753-8704-89ef60b7c0da None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:20.487401 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d404efb-cd3f-4753-8704-89ef60b7c0da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:20.504691 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-2d404efb-cd3f-4753-8704-89ef60b7c0da None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:20.506142 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2d404efb-cd3f-4753-8704-89ef60b7c0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:20.575950 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 439/890] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:20 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 16:03:20.599957 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b553aa1-5712-4a13-9543-11b87c113400 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:20.630501 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:20.707275 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b553aa1-5712-4a13-9543-11b87c113400 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:20.742784 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d869f9-4a8e-456f-ba07-1f952a340bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:20.743877 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d869f9-4a8e-456f-ba07-1f952a340bcb None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:20.744307 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d869f9-4a8e-456f-ba07-1f952a340bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:20.744760 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d869f9-4a8e-456f-ba07-1f952a340bcb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:20.745563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d869f9-4a8e-456f-ba07-1f952a340bcb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:20.747061 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d869f9-4a8e-456f-ba07-1f952a340bcb None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:20.759701 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-87d869f9-4a8e-456f-ba07-1f952a340bcb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 03 16:03:20.759701 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:20.759701 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 16:03:20.759701 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 16:03:20.759701 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 16:03:20.759701 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 16:03:20.759701 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 16:03:20.759701 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:20.759701 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 03 16:03:20.759701 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:20.759701 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 16:03:20.759701 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:20.759701 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:20.759701 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:20.759701 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:20.759701 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:20.760840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:20.760840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:20.760840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:20.760840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:20.760840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:20.760840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:20.760840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:20.760840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 16:03:20.760840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 16:03:20.760840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 16:03:20.760840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 16:03:20.760840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:20.760840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:20.760840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:20.760840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:20.760840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:20.761861 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:20.761861 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:20.761861 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:20.761861 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:20.761861 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:20.761861 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:20.761861 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:20.761861 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:20.761861 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:20.761861 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:20.761861 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:20.761861 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 16:03:20.761861 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:20.761861 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 03 16:03:20.761861 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:20.764428 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 452/891] 10.222.0.41 () {56 vars in 1132 bytes} [Fri Jun 3 16:03:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:20.770989 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d404efb-cd3f-4753-8704-89ef60b7c0da None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:20.817712 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:20.884062 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d404efb-cd3f-4753-8704-89ef60b7c0da None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:20.920012 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196ede3-b0ae-43d9-8ff2-6de812a9df9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:20.920674 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196ede3-b0ae-43d9-8ff2-6de812a9df9d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:20.920959 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196ede3-b0ae-43d9-8ff2-6de812a9df9d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:20.921235 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196ede3-b0ae-43d9-8ff2-6de812a9df9d None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:20.921941 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f196ede3-b0ae-43d9-8ff2-6de812a9df9d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:20.922249 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f196ede3-b0ae-43d9-8ff2-6de812a9df9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:20.923672 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f196ede3-b0ae-43d9-8ff2-6de812a9df9d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:20.928680 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 440/892] 10.222.0.41 () {56 vars in 1126 bytes} [Fri Jun 3 16:03:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:20.934170 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d869f9-4a8e-456f-ba07-1f952a340bcb None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:21.029076 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:21.037632 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d869f9-4a8e-456f-ba07-1f952a340bcb None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:21.073032 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dbf0c8-9d82-4ff9-a5b4-7ad4af5c924e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:21.073701 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dbf0c8-9d82-4ff9-a5b4-7ad4af5c924e None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:21.073992 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dbf0c8-9d82-4ff9-a5b4-7ad4af5c924e None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:21.074267 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dbf0c8-9d82-4ff9-a5b4-7ad4af5c924e None admin] PATH_INFO: `/v3/services` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:21.074974 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dbf0c8-9d82-4ff9-a5b4-7ad4af5c924e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:21.075282 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dbf0c8-9d82-4ff9-a5b4-7ad4af5c924e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:21.076663 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dbf0c8-9d82-4ff9-a5b4-7ad4af5c924e None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:21.082693 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 453/893] 10.222.0.41 () {56 vars in 1126 bytes} [Fri Jun 3 16:03:20 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:21.088397 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196ede3-b0ae-43d9-8ff2-6de812a9df9d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:21.193559 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196ede3-b0ae-43d9-8ff2-6de812a9df9d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:21.203213 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:21.221367 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931cda7d-afa4-4a13-a67d-0d7a460f009c None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:21.221679 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931cda7d-afa4-4a13-a67d-0d7a460f009c None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:21.221943 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931cda7d-afa4-4a13-a67d-0d7a460f009c None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:21.225756 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 454/894] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 16:03:21.228350 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c586e-36cf-4a4c-b560-297b0cae7375 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:21.228640 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c586e-36cf-4a4c-b560-297b0cae7375 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:21.228925 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c586e-36cf-4a4c-b560-297b0cae7375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:21.232116 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b8a963-3f23-4001-8c5f-e4dd0ef9bdbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:21.233098 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8a963-3f23-4001-8c5f-e4dd0ef9bdbc None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:21.233546 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8a963-3f23-4001-8c5f-e4dd0ef9bdbc None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:21.234020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8a963-3f23-4001-8c5f-e4dd0ef9bdbc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:21.234966 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b8a963-3f23-4001-8c5f-e4dd0ef9bdbc None admin] RBAC: Adding query filter params (interface=public, service_id=c9ff376a619a4b818ccfe2c0c8228505, region_id=RegionOne) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:21.235448 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b8a963-3f23-4001-8c5f-e4dd0ef9bdbc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:21.237511 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b8a963-3f23-4001-8c5f-e4dd0ef9bdbc None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:21.243415 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 441/895] 10.222.0.41 () {56 vars in 1251 bytes} [Fri Jun 3 16:03:21 2022] GET /identity/v3/endpoints?service_id=c9ff376a619a4b818ccfe2c0c8228505&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:21.249707 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b8a963-3f23-4001-8c5f-e4dd0ef9bdbc None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:21.251005 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-804c586e-36cf-4a4c-b560-297b0cae7375 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:21.252875 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-804c586e-36cf-4a4c-b560-297b0cae7375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:21.347399 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 455/896] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:21 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 16:03:21.364661 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804c586e-36cf-4a4c-b560-297b0cae7375 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:21.397433 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b8a963-3f23-4001-8c5f-e4dd0ef9bdbc None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:21.450398 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1780b3-14cb-4e53-b617-7bb8e22a9664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:21.452144 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1780b3-14cb-4e53-b617-7bb8e22a9664 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:21.452935 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1780b3-14cb-4e53-b617-7bb8e22a9664 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:21.453418 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1780b3-14cb-4e53-b617-7bb8e22a9664 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:21.454604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1780b3-14cb-4e53-b617-7bb8e22a9664 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:21.456811 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1780b3-14cb-4e53-b617-7bb8e22a9664 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:21.472003 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 442/897] 10.222.0.41 () {56 vars in 1087 bytes} [Fri Jun 3 16:03:21 2022] GET /identity/v3/services => generated 1268 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:03:21.480139 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:21.497623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804c586e-36cf-4a4c-b560-297b0cae7375 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:21.536810 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff6b5e4-5752-4f1b-9fb5-521e511722a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:21.537636 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6b5e4-5752-4f1b-9fb5-521e511722a8 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:21.537920 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6b5e4-5752-4f1b-9fb5-521e511722a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:21.538199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6b5e4-5752-4f1b-9fb5-521e511722a8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:21.550232 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff6b5e4-5752-4f1b-9fb5-521e511722a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:03:21.558681 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-5ff6b5e4-5752-4f1b-9fb5-521e511722a8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 03 16:03:21.558681 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:21.558681 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:21.558681 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:21.558681 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:21.558681 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:21.558681 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:21.558681 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:21.558681 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:21.558681 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:21.558681 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:21.558681 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:21.558681 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:03:21.558681 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:03:21.558681 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:03:21.558681 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:03:21.559776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:03:21.559776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:03:21.559776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:03:21.559776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:03:21.559776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:21.559776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:21.559776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:21.559776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:21.559776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:21.559776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:21.559776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:21.559776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:21.559776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:21.559776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:21.559776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:21.559776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:21.560612 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:21.560612 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:21.560612 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:21.560612 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:21.560612 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:03:21.560612 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:03:21.560612 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:03:21.560612 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:03:21.560612 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:03:21.560612 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:03:21.560612 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 03 16:03:21.560612 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:21.560612 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 456/898] 10.222.0.41 () {56 vars in 1120 bytes} [Fri Jun 3 16:03:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:03:21.568431 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1780b3-14cb-4e53-b617-7bb8e22a9664 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:21.608871 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:21.684100 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1780b3-14cb-4e53-b617-7bb8e22a9664 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:21.726381 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca2614a-3fdd-4437-8500-7d7366c44c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:21.727528 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca2614a-3fdd-4437-8500-7d7366c44c05 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:21.728043 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca2614a-3fdd-4437-8500-7d7366c44c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:21.728518 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca2614a-3fdd-4437-8500-7d7366c44c05 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:21.729466 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca2614a-3fdd-4437-8500-7d7366c44c05 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:21.730005 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca2614a-3fdd-4437-8500-7d7366c44c05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:21.731598 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca2614a-3fdd-4437-8500-7d7366c44c05 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:21.741307 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 443/899] 10.222.0.41 () {56 vars in 1115 bytes} [Fri Jun 3 16:03:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:21.747315 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff6b5e4-5752-4f1b-9fb5-521e511722a8 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:21.770881 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b0acb-771d-4af6-9855-c1643034f049 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:21.771188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b0acb-771d-4af6-9855-c1643034f049 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:21.771484 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b0acb-771d-4af6-9855-c1643034f049 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:21.773171 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 444/900] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:21.778435 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a0681-55b3-458a-9e8f-cc3337bc9e50 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:21.778726 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a0681-55b3-458a-9e8f-cc3337bc9e50 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:21.779006 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a0681-55b3-458a-9e8f-cc3337bc9e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:21.798443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-091a0681-55b3-458a-9e8f-cc3337bc9e50 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:21.800245 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-091a0681-55b3-458a-9e8f-cc3337bc9e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:21.865451 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff6b5e4-5752-4f1b-9fb5-521e511722a8 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:21.885473 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 445/901] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:21 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:03:21.908864 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a0681-55b3-458a-9e8f-cc3337bc9e50 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:21.910061 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac16f1c-28f6-4026-ab67-e8329b70a785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:21.911457 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac16f1c-28f6-4026-ab67-e8329b70a785 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:21.912910 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac16f1c-28f6-4026-ab67-e8329b70a785 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:21.913434 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac16f1c-28f6-4026-ab67-e8329b70a785 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:21.929688 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac16f1c-28f6-4026-ab67-e8329b70a785 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:21.931152 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac16f1c-28f6-4026-ab67-e8329b70a785 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:21.941794 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:21.945721 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:21.961466 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-7ac16f1c-28f6-4026-ab67-e8329b70a785 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:21.961466 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:21.961466 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:21.961466 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:21.961466 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:21.961466 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:21.961466 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:21.961466 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:21.961466 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:21.961466 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:21.961466 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:21.961466 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:21.961466 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:03:21.961466 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:03:21.961466 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:03:21.962423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:03:21.962423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:21.962423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:21.962423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:21.962423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:21.962423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:21.962423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:21.962423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:21.962423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:21.962423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:21.962423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:21.962423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:21.962423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:21.962423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:21.962423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:21.962423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:21.962423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:21.963348 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:03:21.963348 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:03:21.963348 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:21.963348 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:21.965003 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 457/902] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:03:21.972574 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac16f1c-28f6-4026-ab67-e8329b70a785 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:22.023288 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a0681-55b3-458a-9e8f-cc3337bc9e50 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:22.059468 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5c9c29-180f-4cad-b8ca-722281518dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:22.060136 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c9c29-180f-4cad-b8ca-722281518dd9 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:22.060424 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c9c29-180f-4cad-b8ca-722281518dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:22.060710 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c9c29-180f-4cad-b8ca-722281518dd9 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:22.061426 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5c9c29-180f-4cad-b8ca-722281518dd9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:22.062801 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5c9c29-180f-4cad-b8ca-722281518dd9 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:22.076546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac16f1c-28f6-4026-ab67-e8329b70a785 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:22.086038 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 446/903] 10.222.0.41 () {60 vars in 1141 bytes} [Fri Jun 3 16:03:21 2022] POST /identity/v3/services => generated 241 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:03:22.086038 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6d739-a317-4ecb-9916-010f9b4e9e39 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:22.086038 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6d739-a317-4ecb-9916-010f9b4e9e39 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:22.086038 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6d739-a317-4ecb-9916-010f9b4e9e39 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:22.086038 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 447/904] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:22.088444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b87b623-1d06-4ef6-8371-422a9942a640 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:22.088744 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b87b623-1d06-4ef6-8371-422a9942a640 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:22.089026 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b87b623-1d06-4ef6-8371-422a9942a640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:22.106347 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-0b87b623-1d06-4ef6-8371-422a9942a640 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:22.107689 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0b87b623-1d06-4ef6-8371-422a9942a640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:22.137068 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731ad78-39ed-4d9c-8faa-053ce422e488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:22.137751 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731ad78-39ed-4d9c-8faa-053ce422e488 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:22.138036 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731ad78-39ed-4d9c-8faa-053ce422e488 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:22.138316 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731ad78-39ed-4d9c-8faa-053ce422e488 None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:22.139052 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5731ad78-39ed-4d9c-8faa-053ce422e488 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:22.139464 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5731ad78-39ed-4d9c-8faa-053ce422e488 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:22.140931 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5731ad78-39ed-4d9c-8faa-053ce422e488 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:22.155335 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 458/905] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:03:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:22.160776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731ad78-39ed-4d9c-8faa-053ce422e488 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:22.207803 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 448/906] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 16:03:22.226402 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b87b623-1d06-4ef6-8371-422a9942a640 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:22.234008 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:22.257074 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:22.298217 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731ad78-39ed-4d9c-8faa-053ce422e488 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:22.338561 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5442289-f55b-4b6f-8c4e-f56d1b0ce8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:22.339427 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5442289-f55b-4b6f-8c4e-f56d1b0ce8a6 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:22.339715 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5442289-f55b-4b6f-8c4e-f56d1b0ce8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:22.339990 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5442289-f55b-4b6f-8c4e-f56d1b0ce8a6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:22.347975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5442289-f55b-4b6f-8c4e-f56d1b0ce8a6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:22.349370 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5442289-f55b-4b6f-8c4e-f56d1b0ce8a6 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:22.354614 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b87b623-1d06-4ef6-8371-422a9942a640 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:22.365773 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-d5442289-f55b-4b6f-8c4e-f56d1b0ce8a6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 03 16:03:22.365773 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:22.365773 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:22.365773 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:22.365773 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:22.365773 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:22.365773 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:22.365773 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:22.365773 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:22.365773 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:22.365773 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:22.365773 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:22.365773 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:03:22.365773 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:03:22.365773 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:03:22.365773 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:03:22.366717 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:22.366717 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:22.366717 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:03:22.366717 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:03:22.366717 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:03:22.366717 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:03:22.366717 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:22.366717 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:22.366717 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:22.366717 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:22.366717 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:22.366717 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:22.366717 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:22.366717 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:22.366717 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:22.366717 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:22.366717 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:22.366717 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:22.367706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:22.367706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:22.367706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:03:22.367706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:03:22.367706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:03:22.367706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:03:22.367706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:03:22.367706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:03:22.367706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 03 16:03:22.367706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:22.367706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 459/907] 10.222.0.41 () {56 vars in 1117 bytes} [Fri Jun 3 16:03:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:03:22.372780 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5442289-f55b-4b6f-8c4e-f56d1b0ce8a6 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:22.397679 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c4f91b-29ff-4f2b-b58a-781a247f6f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:22.398813 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4f91b-29ff-4f2b-b58a-781a247f6f10 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:22.399470 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4f91b-29ff-4f2b-b58a-781a247f6f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:22.400065 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4f91b-29ff-4f2b-b58a-781a247f6f10 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:22.400796 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c4f91b-29ff-4f2b-b58a-781a247f6f10 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:22.402249 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c4f91b-29ff-4f2b-b58a-781a247f6f10 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:22.415490 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-80c4f91b-29ff-4f2b-b58a-781a247f6f10 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 03 16:03:22.415490 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:22.415490 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 16:03:22.415490 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 16:03:22.415490 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 16:03:22.415490 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 16:03:22.415490 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 16:03:22.415490 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:22.415490 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 03 16:03:22.415490 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:22.415490 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 16:03:22.415490 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:22.415490 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:22.415490 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:22.415490 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:22.415490 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:22.415490 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:22.416510 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:22.416510 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:22.416510 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:22.416510 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:22.416510 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:22.416510 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:22.416510 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 16:03:22.416510 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 16:03:22.416510 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 16:03:22.416510 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 16:03:22.416510 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:22.416510 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:22.416510 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:22.416510 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:22.416510 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:22.416510 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:22.417372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:22.417372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:22.417372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:22.417372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:22.417372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:22.417372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:22.417372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:22.417372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:22.417372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:22.417372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:22.417372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 16:03:22.417372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:22.417372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 03 16:03:22.417372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:22.417372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 449/908] 10.222.0.41 () {56 vars in 1114 bytes} [Fri Jun 3 16:03:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:22.423831 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c4f91b-29ff-4f2b-b58a-781a247f6f10 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:22.489318 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5442289-f55b-4b6f-8c4e-f56d1b0ce8a6 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:22.508539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:22.517977 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:22.535634 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8731ba1-7206-4747-9811-a87de3f34188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:22.536303 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8731ba1-7206-4747-9811-a87de3f34188 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:22.536588 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8731ba1-7206-4747-9811-a87de3f34188 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:22.536869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8731ba1-7206-4747-9811-a87de3f34188 None admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:22.537793 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8731ba1-7206-4747-9811-a87de3f34188 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:22.538015 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8731ba1-7206-4747-9811-a87de3f34188 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:22.539493 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8731ba1-7206-4747-9811-a87de3f34188 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:22.545231 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c4f91b-29ff-4f2b-b58a-781a247f6f10 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:22.562679 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 460/909] 10.222.0.41 () {56 vars in 1149 bytes} [Fri Jun 3 16:03:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:22.564010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8731ba1-7206-4747-9811-a87de3f34188 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:22.589790 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3349e3-b4ed-45e7-bd56-77d5f9ee3473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:22.591781 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3349e3-b4ed-45e7-bd56-77d5f9ee3473 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:22.592288 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3349e3-b4ed-45e7-bd56-77d5f9ee3473 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:22.593024 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3349e3-b4ed-45e7-bd56-77d5f9ee3473 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:22.594350 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3349e3-b4ed-45e7-bd56-77d5f9ee3473 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:22.595092 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3349e3-b4ed-45e7-bd56-77d5f9ee3473 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:22.597632 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3349e3-b4ed-45e7-bd56-77d5f9ee3473 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:22.604398 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 450/910] 10.222.0.41 () {56 vars in 1114 bytes} [Fri Jun 3 16:03:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:22.612291 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3349e3-b4ed-45e7-bd56-77d5f9ee3473 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:22.688899 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8731ba1-7206-4747-9811-a87de3f34188 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:22.723532 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3349e3-b4ed-45e7-bd56-77d5f9ee3473 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:22.739727 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599607f-12b9-4d9b-b0f7-a761abcae84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:22.740971 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599607f-12b9-4d9b-b0f7-a761abcae84b None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:22.741727 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599607f-12b9-4d9b-b0f7-a761abcae84b None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:22.742213 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599607f-12b9-4d9b-b0f7-a761abcae84b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:22.760236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50705da0-6996-49f3-86f3-2c10735335fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:22.760901 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50705da0-6996-49f3-86f3-2c10735335fa None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:22.761189 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50705da0-6996-49f3-86f3-2c10735335fa None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:22.761470 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50705da0-6996-49f3-86f3-2c10735335fa None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:22.762206 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50705da0-6996-49f3-86f3-2c10735335fa None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:22.762489 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50705da0-6996-49f3-86f3-2c10735335fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:22.763975 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50705da0-6996-49f3-86f3-2c10735335fa None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:22.773857 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 451/911] 10.222.0.41 () {56 vars in 1114 bytes} [Fri Jun 3 16:03:22 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:22.775170 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50705da0-6996-49f3-86f3-2c10735335fa None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:22.788858 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:22.798411 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:22.839807 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3599607f-12b9-4d9b-b0f7-a761abcae84b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:22.842476 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3599607f-12b9-4d9b-b0f7-a761abcae84b None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:22.886496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50705da0-6996-49f3-86f3-2c10735335fa None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:22.921197 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7552f0b-ee90-4f5a-8f65-61682effc48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:22.921856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7552f0b-ee90-4f5a-8f65-61682effc48c None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:22.922143 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7552f0b-ee90-4f5a-8f65-61682effc48c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:22.922421 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7552f0b-ee90-4f5a-8f65-61682effc48c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:22.923161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7552f0b-ee90-4f5a-8f65-61682effc48c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:22.925290 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7552f0b-ee90-4f5a-8f65-61682effc48c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:22.951519 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 452/912] 10.222.0.41 () {60 vars in 1144 bytes} [Fri Jun 3 16:03:22 2022] POST /identity/v3/endpoints => generated 346 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:03:22.979131 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 461/913] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:03:22 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 16:03:22.984585 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7552f0b-ee90-4f5a-8f65-61682effc48c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:23.066370 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:23.114352 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7552f0b-ee90-4f5a-8f65-61682effc48c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:23.152387 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f0740-fd53-482a-bba2-03820c5ba670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:23.153782 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f0740-fd53-482a-bba2-03820c5ba670 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:23.154265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f0740-fd53-482a-bba2-03820c5ba670 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:23.154733 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f0740-fd53-482a-bba2-03820c5ba670 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:23.166292 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550f0740-fd53-482a-bba2-03820c5ba670 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:23.168038 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550f0740-fd53-482a-bba2-03820c5ba670 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:23.185400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-550f0740-fd53-482a-bba2-03820c5ba670 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:23.185400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:23.185400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:23.185400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:23.185400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:23.185400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:23.185400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:23.185400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:23.185400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:23.185400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:23.185400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:23.185400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:23.185400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:03:23.185400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:03:23.185400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:03:23.186307 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:03:23.186307 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:23.186307 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:23.186307 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:23.186307 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:23.186307 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:23.186307 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:23.186307 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:23.186307 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:23.186307 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:23.186307 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:23.186307 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:23.186307 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:23.186307 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:23.186307 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:23.186307 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:23.186307 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:23.187201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:03:23.187201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:03:23.187201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:23.187201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:23.188002 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 453/914] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:23.194196 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599607f-12b9-4d9b-b0f7-a761abcae84b None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:23.277117 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:23.301356 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599607f-12b9-4d9b-b0f7-a761abcae84b None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:23.341091 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c8fa6-145d-4427-a48c-ee5e24991f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:23.342249 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c8fa6-145d-4427-a48c-ee5e24991f15 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:23.342771 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c8fa6-145d-4427-a48c-ee5e24991f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:23.343283 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c8fa6-145d-4427-a48c-ee5e24991f15 None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:23.344742 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0c8fa6-145d-4427-a48c-ee5e24991f15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:23.345361 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0c8fa6-145d-4427-a48c-ee5e24991f15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:23.348097 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0c8fa6-145d-4427-a48c-ee5e24991f15 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:23.367519 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 462/915] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:23.369077 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f0740-fd53-482a-bba2-03820c5ba670 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:23.473791 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f0740-fd53-482a-bba2-03820c5ba670 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:23.491661 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:23.512052 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31e780-0420-4ca8-8ee3-1f6fab911d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:23.513144 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31e780-0420-4ca8-8ee3-1f6fab911d8e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:23.513625 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31e780-0420-4ca8-8ee3-1f6fab911d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:23.514096 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31e780-0420-4ca8-8ee3-1f6fab911d8e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:23.522020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31e780-0420-4ca8-8ee3-1f6fab911d8e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:23.523743 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31e780-0420-4ca8-8ee3-1f6fab911d8e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:23.539174 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-0d31e780-0420-4ca8-8ee3-1f6fab911d8e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:03:23.539174 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:23.539174 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:23.539174 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:23.539174 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:23.539174 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:23.539174 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:23.539174 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:23.539174 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:23.539174 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:23.539174 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:23.539174 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:23.539174 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:03:23.539174 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:03:23.539174 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:03:23.540185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:03:23.540185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:23.540185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:23.540185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:23.540185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:23.540185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:23.540185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:23.540185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:23.540185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:23.540185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:23.540185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:23.540185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:23.540185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:23.540185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:23.540185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:23.540185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:23.540185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:23.541078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:03:23.541078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:03:23.541078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:03:23.541078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:03:23.541078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:03:23.541078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:03:23.541078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:03:23.541078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:23.543113 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 454/916] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:03:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:23.549716 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c8fa6-145d-4427-a48c-ee5e24991f15 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:23.655203 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c8fa6-145d-4427-a48c-ee5e24991f15 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:23.684765 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:23.690188 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c376f-49de-4c0c-b8e4-293544228c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:23.690857 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c376f-49de-4c0c-b8e4-293544228c7d None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:23.691535 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c376f-49de-4c0c-b8e4-293544228c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:23.691823 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c376f-49de-4c0c-b8e4-293544228c7d None admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:23.692615 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c376f-49de-4c0c-b8e4-293544228c7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:23.693161 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c376f-49de-4c0c-b8e4-293544228c7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:23.694913 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c376f-49de-4c0c-b8e4-293544228c7d None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:23.711439 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 463/917] 10.222.0.41 () {56 vars in 1148 bytes} [Fri Jun 3 16:03:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:23.714065 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31e780-0420-4ca8-8ee3-1f6fab911d8e None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:23.822222 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31e780-0420-4ca8-8ee3-1f6fab911d8e None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:23.856776 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d519c6-180c-431d-b649-407aa5ed2850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:23.857432 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d519c6-180c-431d-b649-407aa5ed2850 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:23.857716 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d519c6-180c-431d-b649-407aa5ed2850 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:23.857991 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d519c6-180c-431d-b649-407aa5ed2850 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:23.858831 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d519c6-180c-431d-b649-407aa5ed2850 None admin] RBAC: Adding query filter params (role.id=3552bc334c464cd19e06a671ad16af6c, scope.project.id=c274254e510a468e9acbecc6298301cc, user.id=875140cf51d9410da45f8c3805b26204) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:23.859145 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d519c6-180c-431d-b649-407aa5ed2850 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:23.860610 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d519c6-180c-431d-b649-407aa5ed2850 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:23.867135 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 455/918] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:03:23 2022] GET /identity/v3/role_assignments?user.id=875140cf51d9410da45f8c3805b26204&scope.project.id=c274254e510a468e9acbecc6298301cc&role.id=3552bc334c464cd19e06a671ad16af6c => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:23.893216 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333f5f0-35c7-4de9-adc7-9d5a76e870ec None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:23.893773 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333f5f0-35c7-4de9-adc7-9d5a76e870ec None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:23.893773 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333f5f0-35c7-4de9-adc7-9d5a76e870ec None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:23.895262 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 464/919] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:23.901113 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892d914-05d7-4f5a-afe1-071821979587 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:23.901729 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892d914-05d7-4f5a-afe1-071821979587 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:23.902197 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892d914-05d7-4f5a-afe1-071821979587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:23.921391 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-1892d914-05d7-4f5a-afe1-071821979587 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:23.923011 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1892d914-05d7-4f5a-afe1-071821979587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:23.957665 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:24.002114 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 456/920] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:03:24.025338 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d333f5f0-35c7-4de9-adc7-9d5a76e870ec None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:24.138800 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d333f5f0-35c7-4de9-adc7-9d5a76e870ec None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:24.176249 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbfc4e7-f2e2-4f9b-833b-c4c65977d4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:24.177070 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbfc4e7-f2e2-4f9b-833b-c4c65977d4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:24.177354 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbfc4e7-f2e2-4f9b-833b-c4c65977d4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:24.177629 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbfc4e7-f2e2-4f9b-833b-c4c65977d4f2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:24.178342 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbfc4e7-f2e2-4f9b-833b-c4c65977d4f2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:24.179777 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbfc4e7-f2e2-4f9b-833b-c4c65977d4f2 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:24.192361 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-8cbfc4e7-f2e2-4f9b-833b-c4c65977d4f2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 03 16:03:24.192361 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:24.192361 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 16:03:24.192361 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 16:03:24.192361 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 16:03:24.192361 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 16:03:24.192361 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 16:03:24.192361 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:24.192361 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 03 16:03:24.192361 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:24.192361 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 16:03:24.192361 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:24.192361 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:24.192361 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:24.192361 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:24.192361 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:24.193288 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:24.193288 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:24.193288 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:24.193288 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:24.193288 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:24.193288 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:24.193288 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:24.193288 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 16:03:24.193288 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 16:03:24.193288 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 16:03:24.193288 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 16:03:24.193288 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:24.193288 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:24.193288 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:24.193288 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:24.193288 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:24.194485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:24.194485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:24.194485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:24.194485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:24.194485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:24.194485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:24.194485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:24.194485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:24.194485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:24.194485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:24.194485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:24.194485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 16:03:24.194485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:24.194485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 03 16:03:24.194485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:24.194485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 465/921] 10.222.0.41 () {56 vars in 1132 bytes} [Fri Jun 3 16:03:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:24.199907 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892d914-05d7-4f5a-afe1-071821979587 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:24.304758 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892d914-05d7-4f5a-afe1-071821979587 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:24.339931 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694bc64-4ba7-4344-a72f-5469af078d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:24.340594 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694bc64-4ba7-4344-a72f-5469af078d02 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:24.340878 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694bc64-4ba7-4344-a72f-5469af078d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:24.341152 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694bc64-4ba7-4344-a72f-5469af078d02 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:24.341853 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f694bc64-4ba7-4344-a72f-5469af078d02 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:24.342159 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f694bc64-4ba7-4344-a72f-5469af078d02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:24.343555 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f694bc64-4ba7-4344-a72f-5469af078d02 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:24.348729 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 457/922] 10.222.0.41 () {56 vars in 1126 bytes} [Fri Jun 3 16:03:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:24.354125 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbfc4e7-f2e2-4f9b-833b-c4c65977d4f2 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:24.415128 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:24.462225 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbfc4e7-f2e2-4f9b-833b-c4c65977d4f2 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:24.482750 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d2f00-8ada-49b8-84a8-228116f233c7 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:24.483046 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d2f00-8ada-49b8-84a8-228116f233c7 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:24.483322 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d2f00-8ada-49b8-84a8-228116f233c7 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:24.484947 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 458/923] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 16:03:24.490514 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19030b8-d18b-4ef7-9c5c-4600f78b8af6 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:24.490805 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19030b8-d18b-4ef7-9c5c-4600f78b8af6 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:24.491085 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19030b8-d18b-4ef7-9c5c-4600f78b8af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:24.503365 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26378932-0186-45fa-8e64-2a60e4e6db44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:24.504451 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26378932-0186-45fa-8e64-2a60e4e6db44 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:24.504938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26378932-0186-45fa-8e64-2a60e4e6db44 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:24.505471 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26378932-0186-45fa-8e64-2a60e4e6db44 None admin] PATH_INFO: `/v3/services` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:24.506384 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26378932-0186-45fa-8e64-2a60e4e6db44 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:24.506905 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26378932-0186-45fa-8e64-2a60e4e6db44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:24.508503 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26378932-0186-45fa-8e64-2a60e4e6db44 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:24.509895 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-d19030b8-d18b-4ef7-9c5c-4600f78b8af6 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:24.511367 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d19030b8-d18b-4ef7-9c5c-4600f78b8af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:24.514975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 466/924] 10.222.0.41 () {56 vars in 1126 bytes} [Fri Jun 3 16:03:24 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:24.524364 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26378932-0186-45fa-8e64-2a60e4e6db44 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:24.539148 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:24.594662 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 459/925] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 16:03:24.617946 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19030b8-d18b-4ef7-9c5c-4600f78b8af6 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:24.646879 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26378932-0186-45fa-8e64-2a60e4e6db44 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:24.669089 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:24.688021 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7c5d51-b24d-4ea9-98f4-505e73ef7413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:24.688689 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c5d51-b24d-4ea9-98f4-505e73ef7413 None admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:24.689010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c5d51-b24d-4ea9-98f4-505e73ef7413 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:24.689296 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c5d51-b24d-4ea9-98f4-505e73ef7413 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:24.690042 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7c5d51-b24d-4ea9-98f4-505e73ef7413 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:24.691455 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7c5d51-b24d-4ea9-98f4-505e73ef7413 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:24.717472 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 467/926] 10.222.0.41 () {60 vars in 1144 bytes} [Fri Jun 3 16:03:24 2022] POST /identity/v3/endpoints => generated 347 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:03:24.759088 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19030b8-d18b-4ef7-9c5c-4600f78b8af6 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:24.809923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b7877-6281-4aa5-8234-efff77e71b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:24.811164 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b7877-6281-4aa5-8234-efff77e71b64 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:24.811681 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b7877-6281-4aa5-8234-efff77e71b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:24.812159 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b7877-6281-4aa5-8234-efff77e71b64 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:24.813137 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4b7877-6281-4aa5-8234-efff77e71b64 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:24.814832 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4b7877-6281-4aa5-8234-efff77e71b64 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:24.828504 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-0c4b7877-6281-4aa5-8234-efff77e71b64 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 16:03:24.828504 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:24.828504 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 16:03:24.828504 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 16:03:24.828504 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 16:03:24.828504 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 16:03:24.828504 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 16:03:24.828504 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:24.828504 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 16:03:24.828504 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:24.828504 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 16:03:24.828504 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:24.828504 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:24.828504 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:24.828504 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:24.828504 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:24.828504 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:24.829554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:24.829554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:24.829554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:24.829554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:24.829554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:24.829554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:24.829554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 16:03:24.829554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 16:03:24.829554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 16:03:24.829554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 16:03:24.829554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:24.829554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:24.829554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:24.829554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:24.829554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:24.829554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:24.830413 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:24.830413 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:24.830413 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:24.830413 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:24.830413 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:24.830413 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:24.830413 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:24.830413 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:24.830413 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:24.830413 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:24.830413 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 16:03:24.830413 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:24.830413 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 16:03:24.830413 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:24.833454 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 460/927] 10.222.0.41 () {56 vars in 1126 bytes} [Fri Jun 3 16:03:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:03:24.840389 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7c5d51-b24d-4ea9-98f4-505e73ef7413 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:24.955690 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7c5d51-b24d-4ea9-98f4-505e73ef7413 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:24.991056 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3628b-c9c0-40e6-af5f-dba3f166e2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:24.991767 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3628b-c9c0-40e6-af5f-dba3f166e2ed None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:24.992056 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3628b-c9c0-40e6-af5f-dba3f166e2ed None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:24.992331 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3628b-c9c0-40e6-af5f-dba3f166e2ed None admin] PATH_INFO: `/v3/services` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:24.993039 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3628b-c9c0-40e6-af5f-dba3f166e2ed None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:24.993347 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3628b-c9c0-40e6-af5f-dba3f166e2ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:24.994704 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3628b-c9c0-40e6-af5f-dba3f166e2ed None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:25.000164 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 468/928] 10.222.0.41 () {56 vars in 1122 bytes} [Fri Jun 3 16:03:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:25.007073 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b7877-6281-4aa5-8234-efff77e71b64 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:25.033482 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:25.118684 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b7877-6281-4aa5-8234-efff77e71b64 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:25.154703 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2ed03-4f77-48a8-92fc-832ee2b16a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:25.155675 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2ed03-4f77-48a8-92fc-832ee2b16a61 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:25.156102 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2ed03-4f77-48a8-92fc-832ee2b16a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:25.156523 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2ed03-4f77-48a8-92fc-832ee2b16a61 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:25.157374 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf2ed03-4f77-48a8-92fc-832ee2b16a61 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:25.157863 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf2ed03-4f77-48a8-92fc-832ee2b16a61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:25.159425 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf2ed03-4f77-48a8-92fc-832ee2b16a61 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:25.165078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 461/929] 10.222.0.41 () {56 vars in 1122 bytes} [Fri Jun 3 16:03:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:25.171003 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3628b-c9c0-40e6-af5f-dba3f166e2ed None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:25.216240 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:25.279687 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3628b-c9c0-40e6-af5f-dba3f166e2ed None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:25.313912 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add4192-71d0-4a32-9dca-97f92f26d518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:25.315116 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add4192-71d0-4a32-9dca-97f92f26d518 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:25.315431 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add4192-71d0-4a32-9dca-97f92f26d518 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:25.315709 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add4192-71d0-4a32-9dca-97f92f26d518 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:25.316488 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7add4192-71d0-4a32-9dca-97f92f26d518 None admin] RBAC: Adding query filter params (interface=public, service_id=f4b6666fa4914674b9fe8de916d66420, region_id=RegionOne) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:25.316799 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7add4192-71d0-4a32-9dca-97f92f26d518 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:25.318153 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7add4192-71d0-4a32-9dca-97f92f26d518 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:25.323691 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 469/930] 10.222.0.41 () {56 vars in 1251 bytes} [Fri Jun 3 16:03:25 2022] GET /identity/v3/endpoints?service_id=f4b6666fa4914674b9fe8de916d66420&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:25.329110 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2ed03-4f77-48a8-92fc-832ee2b16a61 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:25.395570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f976478-d147-44ea-861b-ac17466e05f9 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:25.395924 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f976478-d147-44ea-861b-ac17466e05f9 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:25.396142 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f976478-d147-44ea-861b-ac17466e05f9 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:25.397759 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 470/931] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 16:03:25.402647 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e25b4-2b98-4b27-9687-dfacca9718cb None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:25.402935 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e25b4-2b98-4b27-9687-dfacca9718cb None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:25.403507 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e25b4-2b98-4b27-9687-dfacca9718cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:25.420560 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-8a9e25b4-2b98-4b27-9687-dfacca9718cb None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:25.421768 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-8a9e25b4-2b98-4b27-9687-dfacca9718cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:25.427693 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:25.439303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2ed03-4f77-48a8-92fc-832ee2b16a61 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:25.485336 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459946d3-bd06-4906-9607-d62dc21935c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:25.486701 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459946d3-bd06-4906-9607-d62dc21935c9 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:25.487257 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459946d3-bd06-4906-9607-d62dc21935c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:25.487550 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459946d3-bd06-4906-9607-d62dc21935c9 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:25.488783 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459946d3-bd06-4906-9607-d62dc21935c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:25.490263 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459946d3-bd06-4906-9607-d62dc21935c9 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:25.496788 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 462/932] 10.222.0.41 () {56 vars in 1087 bytes} [Fri Jun 3 16:03:25 2022] GET /identity/v3/services => generated 1497 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:03:25.499641 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 471/933] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 16:03:25.518143 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459946d3-bd06-4906-9607-d62dc21935c9 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:25.635889 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459946d3-bd06-4906-9607-d62dc21935c9 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:25.674107 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ccf57-967a-4e9d-b91c-89249243d43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:25.674784 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ccf57-967a-4e9d-b91c-89249243d43a None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:25.675114 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ccf57-967a-4e9d-b91c-89249243d43a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:25.675429 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ccf57-967a-4e9d-b91c-89249243d43a None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:25.676145 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ccf57-967a-4e9d-b91c-89249243d43a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:25.677615 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ccf57-967a-4e9d-b91c-89249243d43a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:25.679024 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: WARNING keystone.server.flask.common [None req-1d9ccf57-967a-4e9d-b91c-89249243d43a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 03 16:03:25.700796 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.sql.core [None req-1d9ccf57-967a-4e9d-b91c-89249243d43a None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 03 16:03:25.700796 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 03 16:03:25.700796 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [parameters: {'id': 'bb8c65c111e547abbb09765e7afd1fa5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 03 16:03:25.700796 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80287) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 03 16:03:25.718528 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-1d9ccf57-967a-4e9d-b91c-89249243d43a None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bb8c65c111e547abbb09765e7afd1fa5.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bb8c65c111e547abbb09765e7afd1fa5. Jun 03 16:03:25.718528 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:25.718528 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 03 16:03:25.718528 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 03 16:03:25.718528 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 03 16:03:25.718528 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 03 16:03:25.718528 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 03 16:03:25.718528 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application result = self._query(query) Jun 03 16:03:25.718528 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 03 16:03:25.718528 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application conn.query(q) Jun 03 16:03:25.718528 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 03 16:03:25.718528 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 03 16:03:25.718528 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 03 16:03:25.719443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application result.read() Jun 03 16:03:25.719443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 03 16:03:25.719443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 03 16:03:25.719443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 03 16:03:25.719443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 03 16:03:25.719443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 03 16:03:25.719443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 03 16:03:25.719443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 03 16:03:25.719443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 03 16:03:25.719443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 03 16:03:25.719443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:25.719443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 03 16:03:25.719443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:25.719443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:25.719443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 03 16:03:25.719443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 03 16:03:25.719443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 03 16:03:25.719443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 03 16:03:25.720426 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 03 16:03:25.720426 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application next(self.gen) Jun 03 16:03:25.720426 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 03 16:03:25.720426 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application yield resource Jun 03 16:03:25.720426 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 03 16:03:25.720426 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application next(self.gen) Jun 03 16:03:25.720426 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 03 16:03:25.720426 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self.session.rollback() Jun 03 16:03:25.720426 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 03 16:03:25.720426 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self.force_reraise() Jun 03 16:03:25.720426 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 03 16:03:25.720426 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise self.value Jun 03 16:03:25.720426 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 03 16:03:25.720426 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 03 16:03:25.720426 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 03 16:03:25.720426 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application session.commit() Jun 03 16:03:25.720426 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit Jun 03 16:03:25.721309 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 03 16:03:25.721309 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 03 16:03:25.721309 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._prepare_impl() Jun 03 16:03:25.721309 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 03 16:03:25.721309 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self.session.flush() Jun 03 16:03:25.721309 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush Jun 03 16:03:25.721309 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._flush(objects) Jun 03 16:03:25.721309 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush Jun 03 16:03:25.721309 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 03 16:03:25.721309 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 03 16:03:25.721309 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application compat.raise_( Jun 03 16:03:25.721309 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ Jun 03 16:03:25.721309 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception Jun 03 16:03:25.721309 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush Jun 03 16:03:25.721309 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application flush_context.execute() Jun 03 16:03:25.721309 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 03 16:03:25.721309 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rec.execute(self) Jun 03 16:03:25.722275 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 03 16:03:25.722275 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 03 16:03:25.722275 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 03 16:03:25.722275 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 03 16:03:25.722275 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 03 16:03:25.722275 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 03 16:03:25.722275 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 03 16:03:25.722275 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 03 16:03:25.722275 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 03 16:03:25.722275 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 03 16:03:25.722275 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 03 16:03:25.722275 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 03 16:03:25.722275 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 03 16:03:25.722275 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 03 16:03:25.722275 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 03 16:03:25.724236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 03 16:03:25.724236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ Jun 03 16:03:25.724236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception Jun 03 16:03:25.724236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 03 16:03:25.724236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 03 16:03:25.724236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 03 16:03:25.724236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 03 16:03:25.724236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 03 16:03:25.724236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application result = self._query(query) Jun 03 16:03:25.724236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 03 16:03:25.724236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application conn.query(q) Jun 03 16:03:25.724236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 03 16:03:25.724236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 03 16:03:25.724236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 03 16:03:25.724236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application result.read() Jun 03 16:03:25.724236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 03 16:03:25.724236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 03 16:03:25.725140 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 03 16:03:25.725140 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 03 16:03:25.725140 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 03 16:03:25.725140 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 03 16:03:25.725140 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 03 16:03:25.725140 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 03 16:03:25.725140 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 03 16:03:25.725140 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 03 16:03:25.725140 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [parameters: {'id': 'bb8c65c111e547abbb09765e7afd1fa5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 03 16:03:25.725140 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 03 16:03:25.725140 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:25.725140 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 16:03:25.725140 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:25.725140 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:25.725140 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 03 16:03:25.725914 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 03 16:03:25.725914 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 03 16:03:25.725914 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 03 16:03:25.725914 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 03 16:03:25.725914 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:25.725914 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 16:03:25.725914 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:25.725914 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:25.725914 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:25.725914 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:25.725914 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:25.725914 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:25.725914 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:25.725914 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:25.725914 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:25.725914 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:25.725914 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:25.726789 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:25.726789 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 03 16:03:25.726789 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 03 16:03:25.726789 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:25.726789 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:25.726789 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 03 16:03:25.726789 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 03 16:03:25.726789 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bb8c65c111e547abbb09765e7afd1fa5. Jun 03 16:03:25.726789 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:25.726789 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 463/934] 10.222.0.41 () {60 vars in 1140 bytes} [Fri Jun 3 16:03:25 2022] POST /identity/v3/projects => generated 262 bytes in 203 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:25.729614 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9e25b4-2b98-4b27-9687-dfacca9718cb None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:25.768308 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:25.839124 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9e25b4-2b98-4b27-9687-dfacca9718cb None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:25.874690 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8928dae-d26f-49fd-90fc-a02901afb0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:25.875869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8928dae-d26f-49fd-90fc-a02901afb0ac None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:25.876355 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8928dae-d26f-49fd-90fc-a02901afb0ac None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:25.876868 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8928dae-d26f-49fd-90fc-a02901afb0ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:25.884396 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8928dae-d26f-49fd-90fc-a02901afb0ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:25.887606 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8928dae-d26f-49fd-90fc-a02901afb0ac None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:25.904318 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-a8928dae-d26f-49fd-90fc-a02901afb0ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:03:25.904318 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:25.904318 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:25.904318 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:25.904318 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:25.904318 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:25.904318 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:25.904318 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:25.904318 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:25.904318 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:25.904318 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:25.904318 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:25.904318 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:03:25.904318 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:03:25.904318 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:03:25.905268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:03:25.905268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:25.905268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:25.905268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:25.905268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:25.905268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:25.905268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:25.905268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:25.905268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:25.905268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:25.905268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:25.905268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:25.905268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:25.905268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:25.905268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:25.905268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:25.905268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:25.906152 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:03:25.906152 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:03:25.906152 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:03:25.906152 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:03:25.906152 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:03:25.906152 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:03:25.906152 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:03:25.906152 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:25.909478 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 472/935] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:03:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:25.916742 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ccf57-967a-4e9d-b91c-89249243d43a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:25.928063 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:26.029660 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ccf57-967a-4e9d-b91c-89249243d43a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:26.063140 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79c775-9768-45f2-967a-2444ecc02b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:26.063870 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79c775-9768-45f2-967a-2444ecc02b4b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:26.064757 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79c775-9768-45f2-967a-2444ecc02b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:26.065037 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79c775-9768-45f2-967a-2444ecc02b4b None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:26.065795 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79c775-9768-45f2-967a-2444ecc02b4b None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:26.066103 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79c775-9768-45f2-967a-2444ecc02b4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:26.067760 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79c775-9768-45f2-967a-2444ecc02b4b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:26.088236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 464/936] 10.222.0.41 () {56 vars in 1112 bytes} [Fri Jun 3 16:03:25 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:26.239089 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ed54f-a067-455d-ad6c-f75b38b208d2 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:26.239652 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ed54f-a067-455d-ad6c-f75b38b208d2 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:26.239844 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ed54f-a067-455d-ad6c-f75b38b208d2 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:26.241488 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 473/937] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:26.249453 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890384d-0809-4c68-97de-f0f8783acde0 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:26.249453 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890384d-0809-4c68-97de-f0f8783acde0 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:26.249453 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890384d-0809-4c68-97de-f0f8783acde0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:26.269371 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-6890384d-0809-4c68-97de-f0f8783acde0 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:26.271064 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6890384d-0809-4c68-97de-f0f8783acde0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:26.336786 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:26.344768 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 465/938] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:03:26.366948 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ed54f-a067-455d-ad6c-f75b38b208d2 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:26.467897 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:26.476546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ed54f-a067-455d-ad6c-f75b38b208d2 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:26.511742 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dad1ca7-ea5b-4981-8201-3127bfc472e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:26.512561 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad1ca7-ea5b-4981-8201-3127bfc472e8 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:26.512844 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad1ca7-ea5b-4981-8201-3127bfc472e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:26.513121 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad1ca7-ea5b-4981-8201-3127bfc472e8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:26.524685 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dad1ca7-ea5b-4981-8201-3127bfc472e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:03:26.533071 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-1dad1ca7-ea5b-4981-8201-3127bfc472e8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 03 16:03:26.533071 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:26.533071 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:26.533071 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:26.533071 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:26.533071 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:26.533071 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:26.533071 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:26.533071 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:26.533071 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:26.533071 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:26.533071 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:26.533071 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:03:26.533071 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:03:26.533071 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:03:26.533071 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:03:26.533999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:03:26.533999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:03:26.533999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:03:26.533999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:03:26.533999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:26.533999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:26.533999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:26.533999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:26.533999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:26.533999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:26.533999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:26.533999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:26.533999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:26.533999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:26.533999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:26.533999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:26.534903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:26.534903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:26.534903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:26.534903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:26.534903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:03:26.534903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:03:26.534903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:03:26.534903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:03:26.534903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:03:26.534903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:03:26.534903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 03 16:03:26.534903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:26.534903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 474/939] 10.222.0.41 () {56 vars in 1120 bytes} [Fri Jun 3 16:03:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:26.540262 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890384d-0809-4c68-97de-f0f8783acde0 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:26.642653 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890384d-0809-4c68-97de-f0f8783acde0 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:26.677677 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d49e0-9ae8-4575-a609-1cbeae9200f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:26.678349 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d49e0-9ae8-4575-a609-1cbeae9200f1 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:26.678629 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d49e0-9ae8-4575-a609-1cbeae9200f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:26.678903 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d49e0-9ae8-4575-a609-1cbeae9200f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:26.679676 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d49e0-9ae8-4575-a609-1cbeae9200f1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:26.679980 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d49e0-9ae8-4575-a609-1cbeae9200f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:26.681403 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d49e0-9ae8-4575-a609-1cbeae9200f1 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:26.690924 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 466/940] 10.222.0.41 () {56 vars in 1115 bytes} [Fri Jun 3 16:03:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:26.696428 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dad1ca7-ea5b-4981-8201-3127bfc472e8 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:26.804302 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dad1ca7-ea5b-4981-8201-3127bfc472e8 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:26.839826 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090895ec-6955-4c54-9209-841404a2b04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:26.840870 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090895ec-6955-4c54-9209-841404a2b04e None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:26.841234 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090895ec-6955-4c54-9209-841404a2b04e None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:26.841521 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090895ec-6955-4c54-9209-841404a2b04e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:26.854138 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090895ec-6955-4c54-9209-841404a2b04e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:26.855675 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090895ec-6955-4c54-9209-841404a2b04e None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:26.875985 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-090895ec-6955-4c54-9209-841404a2b04e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:26.875985 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:26.875985 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:26.875985 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:26.875985 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:26.875985 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:26.875985 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:26.875985 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:26.875985 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:26.875985 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:26.875985 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:26.875985 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:26.875985 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:03:26.875985 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:03:26.875985 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:03:26.877003 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:03:26.877003 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:26.877003 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:26.877003 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:26.877003 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:26.877003 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:26.877003 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:26.877003 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:26.877003 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:26.877003 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:26.877003 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:26.877003 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:26.877003 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:26.877003 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:26.877003 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:26.877003 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:26.877003 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:26.877891 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:03:26.877891 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:03:26.877891 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:26.877891 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:26.877891 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 475/941] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:26.883057 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d49e0-9ae8-4575-a609-1cbeae9200f1 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:26.946170 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:26.992028 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d49e0-9ae8-4575-a609-1cbeae9200f1 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:27.027386 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5631f4e-1ca7-4199-b062-8dc6cd85084b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:27.028682 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5631f4e-1ca7-4199-b062-8dc6cd85084b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:27.029174 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5631f4e-1ca7-4199-b062-8dc6cd85084b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:27.029658 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5631f4e-1ca7-4199-b062-8dc6cd85084b None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:27.030581 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5631f4e-1ca7-4199-b062-8dc6cd85084b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:27.031093 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5631f4e-1ca7-4199-b062-8dc6cd85084b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:27.033225 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5631f4e-1ca7-4199-b062-8dc6cd85084b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:27.046411 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 467/942] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:03:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:27.052319 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090895ec-6955-4c54-9209-841404a2b04e None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:27.061257 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:27.154697 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090895ec-6955-4c54-9209-841404a2b04e None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:27.176816 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa63a0-02ae-490c-ba52-adbe920ae191 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:27.177106 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa63a0-02ae-490c-ba52-adbe920ae191 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:27.177388 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa63a0-02ae-490c-ba52-adbe920ae191 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:27.179011 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 468/943] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:27.183747 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b0642-a597-4d5d-a474-02e79c31788d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:27.184035 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b0642-a597-4d5d-a474-02e79c31788d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:27.184317 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b0642-a597-4d5d-a474-02e79c31788d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:27.189727 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05c28b1-35ba-45fa-b81b-f070d478bae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:27.190581 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c28b1-35ba-45fa-b81b-f070d478bae6 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:27.190863 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c28b1-35ba-45fa-b81b-f070d478bae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:27.191134 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c28b1-35ba-45fa-b81b-f070d478bae6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:27.198951 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05c28b1-35ba-45fa-b81b-f070d478bae6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:27.200368 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05c28b1-35ba-45fa-b81b-f070d478bae6 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:27.203474 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-fb9b0642-a597-4d5d-a474-02e79c31788d None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:27.205006 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-fb9b0642-a597-4d5d-a474-02e79c31788d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:27.216378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-c05c28b1-35ba-45fa-b81b-f070d478bae6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 03 16:03:27.216378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:27.216378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:27.216378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:27.216378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:27.216378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:27.216378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:27.216378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:27.216378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:27.216378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:27.216378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:27.216378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:27.216378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:03:27.216378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:03:27.216378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:03:27.216378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:03:27.217307 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:27.217307 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:27.217307 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:03:27.217307 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:03:27.217307 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:03:27.217307 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:03:27.217307 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:27.217307 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:27.217307 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:27.217307 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:27.217307 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:27.217307 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:27.217307 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:27.217307 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:27.217307 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:27.217307 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:27.217307 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:27.217307 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:27.218239 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:27.218239 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:27.218239 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:03:27.218239 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:03:27.218239 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:03:27.218239 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:03:27.218239 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:03:27.218239 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:03:27.218239 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 03 16:03:27.218239 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:27.218239 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 476/944] 10.222.0.41 () {56 vars in 1117 bytes} [Fri Jun 3 16:03:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:27.228400 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05c28b1-35ba-45fa-b81b-f070d478bae6 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:27.290855 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 469/945] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 16:03:27.307598 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:27.311797 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b0642-a597-4d5d-a474-02e79c31788d None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:27.357542 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05c28b1-35ba-45fa-b81b-f070d478bae6 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:27.370849 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:27.403355 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2022d1-2a8e-45af-ba8e-0cf7c0b26a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:27.404037 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2022d1-2a8e-45af-ba8e-0cf7c0b26a05 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:27.404321 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2022d1-2a8e-45af-ba8e-0cf7c0b26a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:27.404598 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2022d1-2a8e-45af-ba8e-0cf7c0b26a05 None admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:27.405445 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2022d1-2a8e-45af-ba8e-0cf7c0b26a05 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:27.405753 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2022d1-2a8e-45af-ba8e-0cf7c0b26a05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:27.407772 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2022d1-2a8e-45af-ba8e-0cf7c0b26a05 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:27.433663 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b0642-a597-4d5d-a474-02e79c31788d None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:27.437021 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 477/946] 10.222.0.41 () {56 vars in 1149 bytes} [Fri Jun 3 16:03:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:27.439860 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2022d1-2a8e-45af-ba8e-0cf7c0b26a05 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:27.474077 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc81d1-bb2d-4260-bfeb-619f61f3e015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:27.475930 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc81d1-bb2d-4260-bfeb-619f61f3e015 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:27.476437 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc81d1-bb2d-4260-bfeb-619f61f3e015 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:27.477157 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc81d1-bb2d-4260-bfeb-619f61f3e015 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:27.478661 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfc81d1-bb2d-4260-bfeb-619f61f3e015 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:27.480287 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfc81d1-bb2d-4260-bfeb-619f61f3e015 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:27.497443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-3bfc81d1-bb2d-4260-bfeb-619f61f3e015 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 03 16:03:27.497443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:27.497443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 16:03:27.497443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 16:03:27.497443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 16:03:27.497443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 16:03:27.497443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 16:03:27.497443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:27.497443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 03 16:03:27.497443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:27.497443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 16:03:27.497443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:27.497443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:27.497443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:27.497443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:27.497443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:27.497443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:27.498483 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:27.498483 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:27.498483 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:27.498483 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:27.498483 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:27.498483 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:27.498483 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 16:03:27.498483 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 16:03:27.498483 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 16:03:27.498483 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 16:03:27.498483 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:27.498483 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:27.498483 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:27.498483 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:27.498483 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:27.498483 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:27.499343 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:27.499343 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:27.499343 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:27.499343 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:27.499343 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:27.499343 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:27.499343 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:27.499343 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:27.499343 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:27.499343 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:27.499343 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 16:03:27.499343 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:27.499343 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 03 16:03:27.499343 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:27.506391 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 470/947] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:03:27 2022] GET /identity/v3/services/compute => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:27.507108 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc81d1-bb2d-4260-bfeb-619f61f3e015 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:27.570669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2022d1-2a8e-45af-ba8e-0cf7c0b26a05 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:27.601764 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:27.614185 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77bd4d-4d12-41d2-a641-265533cf9838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:27.614981 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77bd4d-4d12-41d2-a641-265533cf9838 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:27.615265 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77bd4d-4d12-41d2-a641-265533cf9838 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:27.615571 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77bd4d-4d12-41d2-a641-265533cf9838 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:27.626144 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d77bd4d-4d12-41d2-a641-265533cf9838 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:27.627669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d77bd4d-4d12-41d2-a641-265533cf9838 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:27.646680 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc81d1-bb2d-4260-bfeb-619f61f3e015 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:27.646986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-6d77bd4d-4d12-41d2-a641-265533cf9838 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:27.646986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:27.646986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:27.646986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:27.646986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:27.646986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:27.646986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:27.646986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:27.646986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:27.646986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:27.646986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:27.646986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:27.646986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:03:27.646986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:03:27.646986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:03:27.647869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:03:27.647869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:27.647869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:27.647869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:27.647869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:27.647869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:27.647869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:27.647869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:27.647869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:27.647869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:27.647869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:27.647869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:27.647869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:27.647869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:27.647869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:27.647869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:27.647869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:27.648740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:03:27.648740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:03:27.648740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:27.648740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:27.653386 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 478/948] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:03:27.654861 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77bd4d-4d12-41d2-a641-265533cf9838 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:27.668568 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:27.700395 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be576f0f-7aad-4dae-9739-461408f786ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:27.701444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be576f0f-7aad-4dae-9739-461408f786ca None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:27.702390 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be576f0f-7aad-4dae-9739-461408f786ca None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:27.702853 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be576f0f-7aad-4dae-9739-461408f786ca None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:27.703790 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be576f0f-7aad-4dae-9739-461408f786ca None admin] RBAC: Adding query filter params (name=compute) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:27.704682 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be576f0f-7aad-4dae-9739-461408f786ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:27.706963 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be576f0f-7aad-4dae-9739-461408f786ca None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:27.718838 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 471/949] 10.222.0.41 () {56 vars in 1112 bytes} [Fri Jun 3 16:03:27 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:27.720764 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be576f0f-7aad-4dae-9739-461408f786ca None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:27.786125 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77bd4d-4d12-41d2-a641-265533cf9838 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:27.827866 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be576f0f-7aad-4dae-9739-461408f786ca None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:27.833783 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ccb13-298a-4e1c-9462-daca0573ca8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:27.834619 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ccb13-298a-4e1c-9462-daca0573ca8c None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:27.835047 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ccb13-298a-4e1c-9462-daca0573ca8c None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:27.835510 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ccb13-298a-4e1c-9462-daca0573ca8c None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:27.836877 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56ccb13-298a-4e1c-9462-daca0573ca8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:27.837355 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56ccb13-298a-4e1c-9462-daca0573ca8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:27.839317 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56ccb13-298a-4e1c-9462-daca0573ca8c None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:27.863325 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 479/950] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:03:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:27.866043 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ccb13-298a-4e1c-9462-daca0573ca8c None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:27.888228 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8bf84e-2b47-4974-95bf-54f5fad0f638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:27.889544 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8bf84e-2b47-4974-95bf-54f5fad0f638 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:27.890037 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8bf84e-2b47-4974-95bf-54f5fad0f638 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:27.890619 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8bf84e-2b47-4974-95bf-54f5fad0f638 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:27.891564 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8bf84e-2b47-4974-95bf-54f5fad0f638 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:27.892573 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8bf84e-2b47-4974-95bf-54f5fad0f638 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:27.894370 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8bf84e-2b47-4974-95bf-54f5fad0f638 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:27.908289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 472/951] 10.222.0.41 () {56 vars in 1112 bytes} [Fri Jun 3 16:03:27 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:27.913733 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:27.988537 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ccb13-298a-4e1c-9462-daca0573ca8c None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:28.001240 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f932836d-3de0-474e-9df3-3187da16a0ba None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:28.001821 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f932836d-3de0-474e-9df3-3187da16a0ba None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:28.001821 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f932836d-3de0-474e-9df3-3187da16a0ba None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:28.003562 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 473/952] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 16:03:28.009269 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f1d9a-13ee-4a66-bd86-77a70c8bfc9b None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:28.009613 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f1d9a-13ee-4a66-bd86-77a70c8bfc9b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:28.009876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f1d9a-13ee-4a66-bd86-77a70c8bfc9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:28.029620 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-4c3f1d9a-13ee-4a66-bd86-77a70c8bfc9b None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:28.030978 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4c3f1d9a-13ee-4a66-bd86-77a70c8bfc9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:28.037046 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ca83d-89f8-486a-95aa-f3930cd50a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:28.038247 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ca83d-89f8-486a-95aa-f3930cd50a51 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:28.038803 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ca83d-89f8-486a-95aa-f3930cd50a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:28.039322 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ca83d-89f8-486a-95aa-f3930cd50a51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:28.047876 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ca83d-89f8-486a-95aa-f3930cd50a51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:28.049782 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ca83d-89f8-486a-95aa-f3930cd50a51 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:28.067010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-f83ca83d-89f8-486a-95aa-f3930cd50a51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:03:28.067010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:28.067010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:28.067010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:28.067010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:28.067010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:28.067010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:28.067010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:28.067010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:28.067010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:28.067010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:28.067010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:28.067010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:03:28.067010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:03:28.067010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:03:28.068067 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:03:28.068067 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:28.068067 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:28.068067 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:28.068067 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:28.068067 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:28.068067 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:28.068067 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:28.068067 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:28.068067 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:28.068067 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:28.068067 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:28.068067 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:28.068067 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:28.068067 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:28.068067 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:28.068067 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:28.069039 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:03:28.069039 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:03:28.069039 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:03:28.069039 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:03:28.069039 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:03:28.069039 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:03:28.069039 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:03:28.069039 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:28.071444 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 480/953] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:03:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:03:28.073669 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:28.078017 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ca83d-89f8-486a-95aa-f3930cd50a51 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:28.122068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 474/954] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:03:28.138493 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f1d9a-13ee-4a66-bd86-77a70c8bfc9b None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:28.206319 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:28.211952 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ca83d-89f8-486a-95aa-f3930cd50a51 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:28.250158 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6a7f5-1d2f-4df1-aea2-4449653a00c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:28.250158 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6a7f5-1d2f-4df1-aea2-4449653a00c0 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:28.250158 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6a7f5-1d2f-4df1-aea2-4449653a00c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:28.250158 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6a7f5-1d2f-4df1-aea2-4449653a00c0 None admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:28.250566 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e6a7f5-1d2f-4df1-aea2-4449653a00c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:28.250566 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e6a7f5-1d2f-4df1-aea2-4449653a00c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:28.252002 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e6a7f5-1d2f-4df1-aea2-4449653a00c0 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:28.264205 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 481/955] 10.222.0.41 () {56 vars in 1148 bytes} [Fri Jun 3 16:03:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:28.270420 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6a7f5-1d2f-4df1-aea2-4449653a00c0 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:28.273465 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f1d9a-13ee-4a66-bd86-77a70c8bfc9b None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:28.310234 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c4c40d-2633-4dc2-8551-5132b983cf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:28.311284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c4c40d-2633-4dc2-8551-5132b983cf62 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:28.311741 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c4c40d-2633-4dc2-8551-5132b983cf62 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:28.312165 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c4c40d-2633-4dc2-8551-5132b983cf62 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:28.313053 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c4c40d-2633-4dc2-8551-5132b983cf62 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:28.314689 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c4c40d-2633-4dc2-8551-5132b983cf62 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:28.327433 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-93c4c40d-2633-4dc2-8551-5132b983cf62 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 16:03:28.327433 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:28.327433 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 16:03:28.327433 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 16:03:28.327433 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 16:03:28.327433 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 16:03:28.327433 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 16:03:28.327433 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:28.327433 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 16:03:28.327433 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:28.327433 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 16:03:28.327433 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:28.327433 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:28.327433 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:28.327433 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:28.327433 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:28.327433 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:28.328554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:28.328554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:28.328554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:28.328554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:28.328554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:28.328554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:28.328554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 16:03:28.328554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 16:03:28.328554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 16:03:28.328554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 16:03:28.328554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:28.328554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:28.328554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:28.328554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:28.328554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:28.328554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:28.329923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:28.329923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:28.329923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:28.329923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:28.329923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:28.329923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:28.329923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:28.329923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:28.329923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:28.329923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:28.329923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 16:03:28.329923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:28.329923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 16:03:28.329923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:28.337618 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 475/956] 10.222.0.41 () {56 vars in 1126 bytes} [Fri Jun 3 16:03:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:03:28.339153 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c4c40d-2633-4dc2-8551-5132b983cf62 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:28.382516 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:28.390793 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6a7f5-1d2f-4df1-aea2-4449653a00c0 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:28.426594 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b8589-9476-49fb-b24a-46510ccec1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:28.428002 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b8589-9476-49fb-b24a-46510ccec1b0 None admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:28.428507 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b8589-9476-49fb-b24a-46510ccec1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:28.428984 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b8589-9476-49fb-b24a-46510ccec1b0 None admin] PATH_INFO: `/v3/projects/c274254e510a468e9acbecc6298301cc/users/875140cf51d9410da45f8c3805b26204/roles/3552bc334c464cd19e06a671ad16af6c` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:28.431825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092b8589-9476-49fb-b24a-46510ccec1b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=c274254e510a468e9acbecc6298301cc, user_id=875140cf51d9410da45f8c3805b26204, role_id=3552bc334c464cd19e06a671ad16af6c)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:28.433468 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092b8589-9476-49fb-b24a-46510ccec1b0 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:28.453793 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 482/957] 10.222.0.41 () {58 vars in 1439 bytes} [Fri Jun 3 16:03:28 2022] PUT /identity/v3/projects/c274254e510a468e9acbecc6298301cc/users/875140cf51d9410da45f8c3805b26204/roles/3552bc334c464cd19e06a671ad16af6c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:03:28.472008 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c4c40d-2633-4dc2-8551-5132b983cf62 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:28.506097 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dff544-03f3-4462-9ab7-0b6fb54895e5 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:28.506753 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dff544-03f3-4462-9ab7-0b6fb54895e5 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:28.507099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dff544-03f3-4462-9ab7-0b6fb54895e5 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:28.510459 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 483/958] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:28.512351 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586f2a4-29be-4067-a821-a0d1d41c02a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:28.513229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586f2a4-29be-4067-a821-a0d1d41c02a7 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:28.513694 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586f2a4-29be-4067-a821-a0d1d41c02a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:28.514203 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586f2a4-29be-4067-a821-a0d1d41c02a7 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:28.515059 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a751c79-896b-4436-8cbc-0df45ba1ba01 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:28.515186 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586f2a4-29be-4067-a821-a0d1d41c02a7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:28.515419 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a751c79-896b-4436-8cbc-0df45ba1ba01 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:28.515521 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586f2a4-29be-4067-a821-a0d1d41c02a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:28.515706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a751c79-896b-4436-8cbc-0df45ba1ba01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:28.516834 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586f2a4-29be-4067-a821-a0d1d41c02a7 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:28.525623 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 476/959] 10.222.0.41 () {56 vars in 1122 bytes} [Fri Jun 3 16:03:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:28.533170 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586f2a4-29be-4067-a821-a0d1d41c02a7 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:28.538324 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-4a751c79-896b-4436-8cbc-0df45ba1ba01 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:28.539704 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-4a751c79-896b-4436-8cbc-0df45ba1ba01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:28.553046 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:28.627420 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 484/960] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:03:28.650567 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a751c79-896b-4436-8cbc-0df45ba1ba01 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:28.672519 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:28.677745 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586f2a4-29be-4067-a821-a0d1d41c02a7 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:28.718623 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b8f318-9322-4487-8faf-6b53c224fe20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:28.719306 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8f318-9322-4487-8faf-6b53c224fe20 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:28.719607 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8f318-9322-4487-8faf-6b53c224fe20 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:28.719885 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8f318-9322-4487-8faf-6b53c224fe20 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:28.720631 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b8f318-9322-4487-8faf-6b53c224fe20 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:28.720981 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b8f318-9322-4487-8faf-6b53c224fe20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:28.722363 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b8f318-9322-4487-8faf-6b53c224fe20 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:28.727712 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 477/961] 10.222.0.41 () {56 vars in 1122 bytes} [Fri Jun 3 16:03:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:28.736786 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b8f318-9322-4487-8faf-6b53c224fe20 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:28.789084 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a751c79-896b-4436-8cbc-0df45ba1ba01 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:28.826889 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2cfa47-d1e8-4a32-8e8a-60d6949f1f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:28.828033 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2cfa47-d1e8-4a32-8e8a-60d6949f1f48 None admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:28.828519 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2cfa47-d1e8-4a32-8e8a-60d6949f1f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:28.829033 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2cfa47-d1e8-4a32-8e8a-60d6949f1f48 None admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:28.829983 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2cfa47-d1e8-4a32-8e8a-60d6949f1f48 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:28.831634 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2cfa47-d1e8-4a32-8e8a-60d6949f1f48 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:28.833222 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: WARNING keystone.server.flask.common [None req-ab2cfa47-d1e8-4a32-8e8a-60d6949f1f48 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 03 16:03:28.846197 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.sql.core [None req-ab2cfa47-d1e8-4a32-8e8a-60d6949f1f48 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 03 16:03:28.846197 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 03 16:03:28.846197 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [parameters: {'user_id': '3604b572047840ef9dca45be586082a9', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 03 16:03:28.846197 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80286) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 03 16:03:28.847188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b8f318-9322-4487-8faf-6b53c224fe20 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:28.855276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-ab2cfa47-d1e8-4a32-8e8a-60d6949f1f48 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 03 16:03:28.855276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:28.855276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 03 16:03:28.855276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 03 16:03:28.855276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 03 16:03:28.855276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 03 16:03:28.855276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 03 16:03:28.855276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application result = self._query(query) Jun 03 16:03:28.855276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 03 16:03:28.855276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application conn.query(q) Jun 03 16:03:28.855276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 03 16:03:28.855276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 03 16:03:28.855276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 03 16:03:28.855276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application result.read() Jun 03 16:03:28.855276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 03 16:03:28.856220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 03 16:03:28.856220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 03 16:03:28.856220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 03 16:03:28.856220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 03 16:03:28.856220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 03 16:03:28.856220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 03 16:03:28.856220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 03 16:03:28.856220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 03 16:03:28.856220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:28.856220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 03 16:03:28.856220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:28.856220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:28.856220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 03 16:03:28.856220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 03 16:03:28.856220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 03 16:03:28.856220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 03 16:03:28.856220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 03 16:03:28.856220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application next(self.gen) Jun 03 16:03:28.857197 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 03 16:03:28.857197 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application yield resource Jun 03 16:03:28.857197 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 03 16:03:28.857197 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application next(self.gen) Jun 03 16:03:28.857197 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 03 16:03:28.857197 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self.session.rollback() Jun 03 16:03:28.857197 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 03 16:03:28.857197 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self.force_reraise() Jun 03 16:03:28.857197 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 03 16:03:28.857197 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise self.value Jun 03 16:03:28.857197 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 03 16:03:28.857197 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 03 16:03:28.857197 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 03 16:03:28.857197 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application session.commit() Jun 03 16:03:28.857197 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit Jun 03 16:03:28.857197 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 03 16:03:28.857197 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 03 16:03:28.858097 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self._prepare_impl() Jun 03 16:03:28.858097 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 03 16:03:28.858097 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self.session.flush() Jun 03 16:03:28.858097 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush Jun 03 16:03:28.858097 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self._flush(objects) Jun 03 16:03:28.858097 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush Jun 03 16:03:28.858097 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 03 16:03:28.858097 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 03 16:03:28.858097 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application compat.raise_( Jun 03 16:03:28.858097 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ Jun 03 16:03:28.858097 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception Jun 03 16:03:28.858097 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush Jun 03 16:03:28.858097 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application flush_context.execute() Jun 03 16:03:28.858097 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 03 16:03:28.858097 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rec.execute(self) Jun 03 16:03:28.858097 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 03 16:03:28.858097 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 03 16:03:28.859006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 03 16:03:28.859006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 03 16:03:28.859006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 03 16:03:28.859006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 03 16:03:28.859006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 03 16:03:28.859006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 03 16:03:28.859006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 03 16:03:28.859006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 03 16:03:28.859006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 03 16:03:28.859006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 03 16:03:28.859006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 03 16:03:28.859006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 03 16:03:28.859006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 03 16:03:28.859006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 03 16:03:28.859006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ Jun 03 16:03:28.859006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception Jun 03 16:03:28.861704 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 03 16:03:28.861704 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 03 16:03:28.861704 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 03 16:03:28.861704 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 03 16:03:28.861704 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 03 16:03:28.861704 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application result = self._query(query) Jun 03 16:03:28.861704 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 03 16:03:28.861704 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application conn.query(q) Jun 03 16:03:28.861704 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 03 16:03:28.861704 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 03 16:03:28.861704 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 03 16:03:28.861704 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application result.read() Jun 03 16:03:28.861704 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 03 16:03:28.861704 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 03 16:03:28.861704 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 03 16:03:28.861704 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 03 16:03:28.862544 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 03 16:03:28.862544 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 03 16:03:28.862544 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 03 16:03:28.862544 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 03 16:03:28.862544 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 03 16:03:28.862544 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 03 16:03:28.862544 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [parameters: {'user_id': '3604b572047840ef9dca45be586082a9', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 03 16:03:28.862544 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 03 16:03:28.862544 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:28.862544 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 16:03:28.862544 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:28.862544 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:28.862544 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:28.862544 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:28.862544 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:28.864016 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:28.864016 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:28.864016 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:28.864016 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:28.864016 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:28.864016 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:28.864016 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:28.864016 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 03 16:03:28.864016 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 03 16:03:28.864016 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:28.864016 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:28.864016 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:03:28.864016 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:03:28.864016 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:03:28.864016 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:03:28.864016 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 03 16:03:28.864016 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 03 16:03:28.866077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 03 16:03:28.866077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 03 16:03:28.866077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 03 16:03:28.866077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 03 16:03:28.866077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 03 16:03:28.866077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:28.873493 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 485/962] 10.222.0.41 () {60 vars in 1131 bytes} [Fri Jun 3 16:03:28 2022] POST /identity/v3/users => generated 160 bytes in 219 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:03:28.875017 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2cfa47-d1e8-4a32-8e8a-60d6949f1f48 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:28.883549 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bab4c2-b84f-49d7-9d48-644533d4af30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:28.884145 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bab4c2-b84f-49d7-9d48-644533d4af30 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:28.884431 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bab4c2-b84f-49d7-9d48-644533d4af30 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:28.884712 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bab4c2-b84f-49d7-9d48-644533d4af30 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:28.885463 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bab4c2-b84f-49d7-9d48-644533d4af30 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:28.886839 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bab4c2-b84f-49d7-9d48-644533d4af30 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:28.890862 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:28.915341 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 478/963] 10.222.0.41 () {60 vars in 1144 bytes} [Fri Jun 3 16:03:28 2022] POST /identity/v3/endpoints => generated 349 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:03:29.004845 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2cfa47-d1e8-4a32-8e8a-60d6949f1f48 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:29.054025 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd388d2c-ae08-4795-a22c-be2789f881ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:29.054850 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd388d2c-ae08-4795-a22c-be2789f881ba None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:29.055137 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd388d2c-ae08-4795-a22c-be2789f881ba None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:29.055450 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd388d2c-ae08-4795-a22c-be2789f881ba None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:29.064657 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd388d2c-ae08-4795-a22c-be2789f881ba None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:29.066272 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd388d2c-ae08-4795-a22c-be2789f881ba None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:29.082070 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-fd388d2c-ae08-4795-a22c-be2789f881ba None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 03 16:03:29.082070 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:29.082070 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:29.082070 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:29.082070 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:29.082070 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:29.082070 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:29.082070 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:29.082070 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:29.082070 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:29.082070 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:29.082070 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:29.082070 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:03:29.082070 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:03:29.082070 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:03:29.082070 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:03:29.083059 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:29.083059 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:29.083059 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:03:29.083059 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:03:29.083059 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:03:29.083059 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:03:29.083059 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:29.083059 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:29.083059 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:29.083059 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:29.083059 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:29.083059 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:29.083059 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:29.083059 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:29.083059 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:29.083059 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:29.083059 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:29.083059 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:29.084033 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:29.084033 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:29.084033 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:03:29.084033 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:03:29.084033 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:03:29.084033 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:03:29.084033 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:03:29.084033 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:03:29.084033 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 03 16:03:29.084033 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:29.084033 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 486/964] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:03:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:29.090669 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bab4c2-b84f-49d7-9d48-644533d4af30 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:29.112788 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:29.204930 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bab4c2-b84f-49d7-9d48-644533d4af30 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:29.245299 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb382879-01b5-453c-b421-ea1a830f4f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:29.246326 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb382879-01b5-453c-b421-ea1a830f4f3c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:29.246808 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb382879-01b5-453c-b421-ea1a830f4f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:29.247350 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb382879-01b5-453c-b421-ea1a830f4f3c None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:29.248360 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb382879-01b5-453c-b421-ea1a830f4f3c None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:29.248864 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb382879-01b5-453c-b421-ea1a830f4f3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:29.250486 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb382879-01b5-453c-b421-ea1a830f4f3c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:29.290263 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 479/965] 10.222.0.41 () {56 vars in 1101 bytes} [Fri Jun 3 16:03:29 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:30.274181 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a62a11-5fba-474f-924b-b06305a12a24 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:30.274701 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a62a11-5fba-474f-924b-b06305a12a24 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:30.274781 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a62a11-5fba-474f-924b-b06305a12a24 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:30.276151 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 487/966] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:30.281504 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f72dc-2bd0-408d-8287-eda3475f550c None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:30.281879 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f72dc-2bd0-408d-8287-eda3475f550c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:30.282067 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f72dc-2bd0-408d-8287-eda3475f550c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:30.299043 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-9d0f72dc-2bd0-408d-8287-eda3475f550c None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:30.300268 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9d0f72dc-2bd0-408d-8287-eda3475f550c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:30.366536 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 480/967] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:03:30.389177 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a62a11-5fba-474f-924b-b06305a12a24 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:30.457391 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:30.498491 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a62a11-5fba-474f-924b-b06305a12a24 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:30.534498 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b307a2-edad-4327-9081-ae4ef4d68961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:30.534498 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b307a2-edad-4327-9081-ae4ef4d68961 None admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:30.534777 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b307a2-edad-4327-9081-ae4ef4d68961 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:30.535067 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b307a2-edad-4327-9081-ae4ef4d68961 None admin] PATH_INFO: `/v3/services` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:30.535832 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b307a2-edad-4327-9081-ae4ef4d68961 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:30.537183 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b307a2-edad-4327-9081-ae4ef4d68961 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:30.557515 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 488/968] 10.222.0.41 () {60 vars in 1141 bytes} [Fri Jun 3 16:03:30 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:03:30.776328 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ecbb3-ebcd-4243-8aa5-d3c0ab32aa7d None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:30.776688 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ecbb3-ebcd-4243-8aa5-d3c0ab32aa7d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:30.776892 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ecbb3-ebcd-4243-8aa5-d3c0ab32aa7d None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:30.778520 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 481/969] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:30.784685 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda04a8-8396-41cc-a623-4b8806a3240c None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:30.784929 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda04a8-8396-41cc-a623-4b8806a3240c None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:30.785218 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda04a8-8396-41cc-a623-4b8806a3240c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:30.802994 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-6bda04a8-8396-41cc-a623-4b8806a3240c None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:30.804577 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-6bda04a8-8396-41cc-a623-4b8806a3240c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:30.879848 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 489/970] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:30 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:03:30.904178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ecbb3-ebcd-4243-8aa5-d3c0ab32aa7d None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:30.916224 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:31.014800 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ecbb3-ebcd-4243-8aa5-d3c0ab32aa7d None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:31.049501 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af89774-044f-485a-9225-b896aec4cec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:31.050230 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af89774-044f-485a-9225-b896aec4cec0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:31.050515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af89774-044f-485a-9225-b896aec4cec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:31.050788 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af89774-044f-485a-9225-b896aec4cec0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:31.060826 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af89774-044f-485a-9225-b896aec4cec0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:03:31.068902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-4af89774-044f-485a-9225-b896aec4cec0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 03 16:03:31.068902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:31.068902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:31.068902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:31.068902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:31.068902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:31.068902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:31.068902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:31.068902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:31.068902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:31.068902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:31.068902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:31.068902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:03:31.068902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:03:31.068902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:03:31.068902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:03:31.069808 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:03:31.069808 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:03:31.069808 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:03:31.069808 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:03:31.069808 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:31.069808 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:31.069808 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:31.069808 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:31.069808 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:31.069808 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:31.069808 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:31.069808 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:31.069808 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:31.069808 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:31.069808 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:31.069808 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:31.071448 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:31.071448 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:31.071448 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:31.071448 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:31.071448 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:03:31.071448 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:03:31.071448 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:03:31.071448 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:03:31.071448 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:03:31.071448 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:03:31.071448 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 03 16:03:31.071448 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:31.071448 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 482/971] 10.222.0.41 () {56 vars in 1120 bytes} [Fri Jun 3 16:03:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:31.076275 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda04a8-8396-41cc-a623-4b8806a3240c None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:31.127383 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:31.180309 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda04a8-8396-41cc-a623-4b8806a3240c None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:31.213930 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de879d51-1a13-4c15-ba43-58fef91a0b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:31.214609 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de879d51-1a13-4c15-ba43-58fef91a0b7d None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:31.214911 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de879d51-1a13-4c15-ba43-58fef91a0b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:31.215186 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de879d51-1a13-4c15-ba43-58fef91a0b7d None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:31.215994 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de879d51-1a13-4c15-ba43-58fef91a0b7d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:31.216303 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de879d51-1a13-4c15-ba43-58fef91a0b7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:31.217678 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de879d51-1a13-4c15-ba43-58fef91a0b7d None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:31.233723 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 490/972] 10.222.0.41 () {56 vars in 1115 bytes} [Fri Jun 3 16:03:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:31.239543 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af89774-044f-485a-9225-b896aec4cec0 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:31.273941 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8deac-d280-4122-9ad4-65b78db97dd1 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:31.274289 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8deac-d280-4122-9ad4-65b78db97dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:31.274823 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8deac-d280-4122-9ad4-65b78db97dd1 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:31.276124 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 491/973] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 16:03:31.281436 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9dbf0-f586-4ea3-beb7-982561b3c555 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:31.281726 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9dbf0-f586-4ea3-beb7-982561b3c555 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:31.282009 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9dbf0-f586-4ea3-beb7-982561b3c555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:31.299630 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-85a9dbf0-f586-4ea3-beb7-982561b3c555 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:31.300747 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-85a9dbf0-f586-4ea3-beb7-982561b3c555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:31.309715 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:31.353076 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af89774-044f-485a-9225-b896aec4cec0 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:31.375333 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 492/974] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:31 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 16:03:31.390904 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3889b1-393a-4df6-ae50-5f9c4b0e342b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:31.391765 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3889b1-393a-4df6-ae50-5f9c4b0e342b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:31.392139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3889b1-393a-4df6-ae50-5f9c4b0e342b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:31.392418 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3889b1-393a-4df6-ae50-5f9c4b0e342b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:31.398431 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a9dbf0-f586-4ea3-beb7-982561b3c555 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:31.403801 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3889b1-393a-4df6-ae50-5f9c4b0e342b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:31.405271 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3889b1-393a-4df6-ae50-5f9c4b0e342b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:31.423034 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-0c3889b1-393a-4df6-ae50-5f9c4b0e342b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:31.423034 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:31.423034 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:31.423034 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:31.423034 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:31.423034 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:31.423034 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:31.423034 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:31.423034 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:31.423034 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:31.423034 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:31.423034 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:31.423034 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:03:31.423034 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:03:31.423034 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:03:31.424617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:03:31.424617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:31.424617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:31.424617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:31.424617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:31.424617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:31.424617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:31.424617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:31.424617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:31.424617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:31.424617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:31.424617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:31.424617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:31.424617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:31.424617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:31.424617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:31.424617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:31.425515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:03:31.425515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:03:31.425515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:31.425515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:31.427594 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 483/975] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:03:31.430529 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3889b1-393a-4df6-ae50-5f9c4b0e342b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:31.449665 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:31.511246 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a9dbf0-f586-4ea3-beb7-982561b3c555 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:31.540458 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3889b1-393a-4df6-ae50-5f9c4b0e342b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:31.550273 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3ec42a-291c-49c3-8b83-422391d23968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:31.551806 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ec42a-291c-49c3-8b83-422391d23968 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:31.552184 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ec42a-291c-49c3-8b83-422391d23968 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:31.552529 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ec42a-291c-49c3-8b83-422391d23968 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:31.553245 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3ec42a-291c-49c3-8b83-422391d23968 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:31.554600 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3ec42a-291c-49c3-8b83-422391d23968 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:31.558010 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:31.567287 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-cb3ec42a-291c-49c3-8b83-422391d23968 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 16:03:31.567287 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:31.567287 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 16:03:31.567287 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 16:03:31.567287 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 16:03:31.567287 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 16:03:31.567287 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 16:03:31.567287 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:31.567287 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 16:03:31.567287 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:31.567287 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 16:03:31.567287 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:31.567287 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:31.567287 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:31.567287 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:31.567287 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:31.567287 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:31.568277 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:31.568277 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:31.568277 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:31.568277 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:31.568277 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:31.568277 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:31.568277 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 16:03:31.568277 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 16:03:31.568277 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 16:03:31.568277 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 16:03:31.568277 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:31.568277 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:31.568277 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:31.568277 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:31.568277 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:31.568277 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:31.570077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:31.570077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:31.570077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:31.570077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:31.570077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:31.570077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:31.570077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:31.570077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:31.570077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:31.570077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:31.570077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 16:03:31.570077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:31.570077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 16:03:31.570077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:31.570077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 493/976] 10.222.0.41 () {56 vars in 1126 bytes} [Fri Jun 3 16:03:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:03:31.574308 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3ec42a-291c-49c3-8b83-422391d23968 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:31.579925 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85924e7b-a80a-4fd0-ae13-e190235d9505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:31.580602 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85924e7b-a80a-4fd0-ae13-e190235d9505 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:31.580890 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85924e7b-a80a-4fd0-ae13-e190235d9505 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:31.581175 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85924e7b-a80a-4fd0-ae13-e190235d9505 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:31.581909 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85924e7b-a80a-4fd0-ae13-e190235d9505 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:31.582223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85924e7b-a80a-4fd0-ae13-e190235d9505 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:31.584028 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85924e7b-a80a-4fd0-ae13-e190235d9505 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:31.595830 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 484/977] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:03:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:31.601153 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85924e7b-a80a-4fd0-ae13-e190235d9505 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:31.682007 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3ec42a-291c-49c3-8b83-422391d23968 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:31.693505 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:31.707092 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85924e7b-a80a-4fd0-ae13-e190235d9505 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:31.722747 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0411dcd3-9365-4b73-8531-abcea7945a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:31.723656 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411dcd3-9365-4b73-8531-abcea7945a67 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:31.723960 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411dcd3-9365-4b73-8531-abcea7945a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:31.724225 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411dcd3-9365-4b73-8531-abcea7945a67 None admin] PATH_INFO: `/v3/services` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:31.724930 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0411dcd3-9365-4b73-8531-abcea7945a67 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:31.725239 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0411dcd3-9365-4b73-8531-abcea7945a67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:31.726601 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0411dcd3-9365-4b73-8531-abcea7945a67 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:31.731907 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 494/978] 10.222.0.41 () {56 vars in 1122 bytes} [Fri Jun 3 16:03:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:31.737981 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0411dcd3-9365-4b73-8531-abcea7945a67 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:31.742287 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d0592-dcf5-4cd3-8ead-367c9de491a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:31.743096 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d0592-dcf5-4cd3-8ead-367c9de491a5 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:31.743413 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d0592-dcf5-4cd3-8ead-367c9de491a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:31.743696 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d0592-dcf5-4cd3-8ead-367c9de491a5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:31.751039 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182d0592-dcf5-4cd3-8ead-367c9de491a5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:31.752586 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182d0592-dcf5-4cd3-8ead-367c9de491a5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:31.768188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-182d0592-dcf5-4cd3-8ead-367c9de491a5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 03 16:03:31.768188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:31.768188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:31.768188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:31.768188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:31.768188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:31.768188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:31.768188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:31.768188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:31.768188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:31.768188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:31.768188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:31.768188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:03:31.768188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:03:31.768188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:03:31.768188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:03:31.769149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:31.769149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:31.769149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:03:31.769149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:03:31.769149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:03:31.769149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:03:31.769149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:31.769149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:31.769149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:31.769149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:31.769149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:31.769149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:31.769149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:31.769149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:31.769149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:31.769149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:31.769149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:31.769149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:31.770085 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:31.770085 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:31.770085 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:03:31.770085 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:03:31.770085 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:03:31.770085 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:03:31.770085 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:03:31.770085 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:03:31.770085 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 03 16:03:31.770085 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:31.770085 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 485/979] 10.222.0.41 () {56 vars in 1117 bytes} [Fri Jun 3 16:03:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:03:31.775109 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d0592-dcf5-4cd3-8ead-367c9de491a5 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:31.817604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:31.847432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0411dcd3-9365-4b73-8531-abcea7945a67 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:31.882652 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00c4e5-3f4d-421b-adc6-e86ae54d161c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:31.883341 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00c4e5-3f4d-421b-adc6-e86ae54d161c None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:31.883631 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00c4e5-3f4d-421b-adc6-e86ae54d161c None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:31.883908 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00c4e5-3f4d-421b-adc6-e86ae54d161c None admin] PATH_INFO: `/v3/services` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:31.884251 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d0592-dcf5-4cd3-8ead-367c9de491a5 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:31.884612 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb00c4e5-3f4d-421b-adc6-e86ae54d161c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:31.884922 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb00c4e5-3f4d-421b-adc6-e86ae54d161c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:31.886286 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb00c4e5-3f4d-421b-adc6-e86ae54d161c None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:31.891324 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 495/980] 10.222.0.41 () {56 vars in 1122 bytes} [Fri Jun 3 16:03:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:31.897622 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00c4e5-3f4d-421b-adc6-e86ae54d161c None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:31.922805 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2944808-c1d6-4655-8c44-192506bf1acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:31.924200 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2944808-c1d6-4655-8c44-192506bf1acc None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:31.924334 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2944808-c1d6-4655-8c44-192506bf1acc None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:31.924568 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2944808-c1d6-4655-8c44-192506bf1acc None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:31.925481 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2944808-c1d6-4655-8c44-192506bf1acc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:31.925781 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2944808-c1d6-4655-8c44-192506bf1acc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:31.927133 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2944808-c1d6-4655-8c44-192506bf1acc None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:31.934125 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:31.945845 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 486/981] 10.222.0.41 () {56 vars in 1149 bytes} [Fri Jun 3 16:03:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:31.951427 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2944808-c1d6-4655-8c44-192506bf1acc None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:32.003462 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00c4e5-3f4d-421b-adc6-e86ae54d161c None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:32.040781 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a28c9a-3295-4009-bdb6-e5fd3bc049fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:32.041465 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a28c9a-3295-4009-bdb6-e5fd3bc049fe None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:32.041754 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a28c9a-3295-4009-bdb6-e5fd3bc049fe None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:32.042031 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a28c9a-3295-4009-bdb6-e5fd3bc049fe None admin] PATH_INFO: `/v3/endpoints` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:32.042833 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a28c9a-3295-4009-bdb6-e5fd3bc049fe None admin] RBAC: Adding query filter params (interface=admin, service_id=f4b6666fa4914674b9fe8de916d66420, region_id=RegionOne) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:32.043191 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a28c9a-3295-4009-bdb6-e5fd3bc049fe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:32.044587 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a28c9a-3295-4009-bdb6-e5fd3bc049fe None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:32.050064 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 496/982] 10.222.0.41 () {56 vars in 1249 bytes} [Fri Jun 3 16:03:31 2022] GET /identity/v3/endpoints?service_id=f4b6666fa4914674b9fe8de916d66420&interface=admin®ion_id=RegionOne => generated 193 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:32.052139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:32.055759 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a28c9a-3295-4009-bdb6-e5fd3bc049fe None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:32.059634 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2944808-c1d6-4655-8c44-192506bf1acc None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:32.097120 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355fe8d8-e2e7-44b0-b051-2455268c5802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:32.097703 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355fe8d8-e2e7-44b0-b051-2455268c5802 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:32.097993 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355fe8d8-e2e7-44b0-b051-2455268c5802 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:32.098269 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355fe8d8-e2e7-44b0-b051-2455268c5802 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:32.153443 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:32.159109 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a28c9a-3295-4009-bdb6-e5fd3bc049fe None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:32.168196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355fe8d8-e2e7-44b0-b051-2455268c5802 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:32.169658 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355fe8d8-e2e7-44b0-b051-2455268c5802 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:32.195822 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f102e209-4137-40b3-be09-ce6e5490a190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:32.196488 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102e209-4137-40b3-be09-ce6e5490a190 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:32.196772 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102e209-4137-40b3-be09-ce6e5490a190 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:32.197048 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102e209-4137-40b3-be09-ce6e5490a190 None admin] PATH_INFO: `/v3/services` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:32.197718 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f102e209-4137-40b3-be09-ce6e5490a190 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:32.199077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f102e209-4137-40b3-be09-ce6e5490a190 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:32.205644 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 497/983] 10.222.0.41 () {56 vars in 1087 bytes} [Fri Jun 3 16:03:32 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:03:32.279102 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:32.287959 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 487/984] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:03:31 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 16:03:32.293131 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f102e209-4137-40b3-be09-ce6e5490a190 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:32.407992 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f102e209-4137-40b3-be09-ce6e5490a190 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:32.442741 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2235300-73d5-4688-9fab-b863b9ae6d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:32.443569 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2235300-73d5-4688-9fab-b863b9ae6d72 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:32.443851 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2235300-73d5-4688-9fab-b863b9ae6d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:32.444124 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2235300-73d5-4688-9fab-b863b9ae6d72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:32.455115 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2235300-73d5-4688-9fab-b863b9ae6d72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:32.456580 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2235300-73d5-4688-9fab-b863b9ae6d72 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:32.473770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-a2235300-73d5-4688-9fab-b863b9ae6d72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:32.473770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:32.473770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:32.473770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:32.473770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:32.473770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:32.473770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:32.473770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:32.473770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:32.473770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:32.473770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:32.473770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:32.473770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:03:32.473770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:03:32.473770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:03:32.474873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:03:32.474873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:32.474873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:32.474873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:32.474873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:32.474873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:32.474873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:32.474873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:32.474873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:32.474873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:32.474873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:32.474873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:32.474873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:32.474873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:32.474873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:32.474873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:32.474873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:32.475814 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:03:32.475814 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:03:32.475814 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:32.475814 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:32.475814 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 498/985] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:32.482095 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355fe8d8-e2e7-44b0-b051-2455268c5802 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:32.588540 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355fe8d8-e2e7-44b0-b051-2455268c5802 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:32.622720 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d76c76-a0d0-4daf-b419-6ceb7d1066f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:32.623416 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d76c76-a0d0-4daf-b419-6ceb7d1066f7 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:32.623709 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d76c76-a0d0-4daf-b419-6ceb7d1066f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:32.623992 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d76c76-a0d0-4daf-b419-6ceb7d1066f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:32.624717 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d76c76-a0d0-4daf-b419-6ceb7d1066f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:32.625028 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d76c76-a0d0-4daf-b419-6ceb7d1066f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:32.626433 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d76c76-a0d0-4daf-b419-6ceb7d1066f7 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:32.637729 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 488/986] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:03:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:32.643636 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2235300-73d5-4688-9fab-b863b9ae6d72 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:32.661238 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:32.747459 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2235300-73d5-4688-9fab-b863b9ae6d72 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:32.781114 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f2deea-22f8-4837-8235-cbe7c2d8b735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:32.781905 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f2deea-22f8-4837-8235-cbe7c2d8b735 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:32.782187 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f2deea-22f8-4837-8235-cbe7c2d8b735 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:32.783005 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f2deea-22f8-4837-8235-cbe7c2d8b735 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:32.789762 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f2deea-22f8-4837-8235-cbe7c2d8b735 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:32.791268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f2deea-22f8-4837-8235-cbe7c2d8b735 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:32.805155 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-b2f2deea-22f8-4837-8235-cbe7c2d8b735 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:03:32.805155 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:32.805155 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:32.805155 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:32.805155 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:32.805155 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:32.805155 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:32.805155 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:32.805155 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:32.805155 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:32.805155 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:32.805155 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:32.805155 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:03:32.805155 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:03:32.805155 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:03:32.806127 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:03:32.806127 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:32.806127 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:32.806127 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:32.806127 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:32.806127 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:32.806127 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:32.806127 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:32.806127 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:32.806127 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:32.806127 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:32.806127 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:32.806127 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:32.806127 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:32.806127 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:32.806127 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:32.806127 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:32.807032 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:03:32.807032 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:03:32.807032 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:03:32.807032 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:03:32.807032 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:03:32.807032 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:03:32.807032 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:03:32.807032 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:32.807032 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 499/987] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:03:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:03:32.813621 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d76c76-a0d0-4daf-b419-6ceb7d1066f7 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:32.837808 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785aed44-5216-4ba7-805a-ed0567a2ba12 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:32.838184 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785aed44-5216-4ba7-805a-ed0567a2ba12 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:32.838361 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785aed44-5216-4ba7-805a-ed0567a2ba12 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:32.840015 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 500/988] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 16:03:32.844730 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ebf51-06bb-47a5-848f-96cf3e073d49 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:32.845646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ebf51-06bb-47a5-848f-96cf3e073d49 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:32.845937 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ebf51-06bb-47a5-848f-96cf3e073d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:32.862740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-bf9ebf51-06bb-47a5-848f-96cf3e073d49 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:32.864025 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-bf9ebf51-06bb-47a5-848f-96cf3e073d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:32.878524 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:32.926613 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d76c76-a0d0-4daf-b419-6ceb7d1066f7 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:32.933940 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 501/989] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:32 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 16:03:32.956967 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ebf51-06bb-47a5-848f-96cf3e073d49 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:32.960839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:32.964934 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f7750-fc44-4629-ae87-e519ef7103e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:32.965602 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f7750-fc44-4629-ae87-e519ef7103e8 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:32.965890 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f7750-fc44-4629-ae87-e519ef7103e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:32.966173 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f7750-fc44-4629-ae87-e519ef7103e8 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:32.966973 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f7750-fc44-4629-ae87-e519ef7103e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:32.967304 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f7750-fc44-4629-ae87-e519ef7103e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:32.968759 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f7750-fc44-4629-ae87-e519ef7103e8 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:32.981269 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 489/990] 10.222.0.41 () {56 vars in 1148 bytes} [Fri Jun 3 16:03:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:32.987109 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f7750-fc44-4629-ae87-e519ef7103e8 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:33.067190 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ebf51-06bb-47a5-848f-96cf3e073d49 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:33.092576 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f7750-fc44-4629-ae87-e519ef7103e8 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:33.102005 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c8c79e-9c80-4248-b142-499d29bbca2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:33.102798 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8c79e-9c80-4248-b142-499d29bbca2b None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:33.103079 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8c79e-9c80-4248-b142-499d29bbca2b None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:33.103384 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8c79e-9c80-4248-b142-499d29bbca2b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:33.104025 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c8c79e-9c80-4248-b142-499d29bbca2b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:33.105401 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c8c79e-9c80-4248-b142-499d29bbca2b None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:33.109275 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:33.118551 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-07c8c79e-9c80-4248-b142-499d29bbca2b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 03 16:03:33.118551 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:33.118551 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 16:03:33.118551 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 16:03:33.118551 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 16:03:33.118551 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 16:03:33.118551 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 16:03:33.118551 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:33.118551 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 03 16:03:33.118551 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:33.118551 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 16:03:33.118551 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:33.118551 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:33.118551 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:33.118551 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:33.118551 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:33.118551 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:33.119567 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:33.119567 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:33.119567 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:33.119567 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:33.119567 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:33.119567 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:33.119567 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 16:03:33.119567 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 16:03:33.119567 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 16:03:33.119567 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 16:03:33.119567 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:33.119567 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:33.119567 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:33.119567 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:33.119567 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:33.119567 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:33.120420 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:33.120420 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:33.120420 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:33.120420 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:33.120420 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:33.120420 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:33.120420 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:33.120420 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:33.120420 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:33.120420 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:33.120420 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 16:03:33.120420 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:33.120420 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 03 16:03:33.120420 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:33.120420 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 502/991] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:03:32 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:03:33.125465 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c8c79e-9c80-4248-b142-499d29bbca2b None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:33.131197 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031da7a8-66fc-4ec9-9286-6b6816166e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:33.131899 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031da7a8-66fc-4ec9-9286-6b6816166e15 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:33.132187 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031da7a8-66fc-4ec9-9286-6b6816166e15 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:33.132467 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031da7a8-66fc-4ec9-9286-6b6816166e15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:33.133304 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031da7a8-66fc-4ec9-9286-6b6816166e15 None admin] RBAC: Adding query filter params (role.id=3552bc334c464cd19e06a671ad16af6c, scope.project.id=c274254e510a468e9acbecc6298301cc, user.id=875140cf51d9410da45f8c3805b26204) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:33.133619 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031da7a8-66fc-4ec9-9286-6b6816166e15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:33.135051 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031da7a8-66fc-4ec9-9286-6b6816166e15 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:33.141106 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 490/992] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:03:32 2022] GET /identity/v3/role_assignments?user.id=875140cf51d9410da45f8c3805b26204&scope.project.id=c274254e510a468e9acbecc6298301cc&role.id=3552bc334c464cd19e06a671ad16af6c => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:33.207944 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:33.255397 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c8c79e-9c80-4248-b142-499d29bbca2b None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:33.292776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede452d-fea7-446c-bb7d-94d3099d6bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:33.293447 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede452d-fea7-446c-bb7d-94d3099d6bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:33.293978 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede452d-fea7-446c-bb7d-94d3099d6bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:33.293978 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede452d-fea7-446c-bb7d-94d3099d6bb7 None admin] PATH_INFO: `/v3/services` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:33.294695 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede452d-fea7-446c-bb7d-94d3099d6bb7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:33.295002 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede452d-fea7-446c-bb7d-94d3099d6bb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:33.296488 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede452d-fea7-446c-bb7d-94d3099d6bb7 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:33.307196 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 503/993] 10.222.0.41 () {56 vars in 1112 bytes} [Fri Jun 3 16:03:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:33.308374 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031da7a8-66fc-4ec9-9286-6b6816166e15 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:33.414570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031da7a8-66fc-4ec9-9286-6b6816166e15 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:33.448810 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f63f5-ae64-494a-be4e-b3cac013aa6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:33.449468 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f63f5-ae64-494a-be4e-b3cac013aa6f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:33.449754 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f63f5-ae64-494a-be4e-b3cac013aa6f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:33.450033 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f63f5-ae64-494a-be4e-b3cac013aa6f None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:33.450743 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49f63f5-ae64-494a-be4e-b3cac013aa6f None admin] RBAC: Adding query filter params (type=compute) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:33.451078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49f63f5-ae64-494a-be4e-b3cac013aa6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:33.452481 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49f63f5-ae64-494a-be4e-b3cac013aa6f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:33.457338 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 491/994] 10.222.0.41 () {56 vars in 1112 bytes} [Fri Jun 3 16:03:33 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:33.464307 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede452d-fea7-446c-bb7d-94d3099d6bb7 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:33.571899 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede452d-fea7-446c-bb7d-94d3099d6bb7 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:33.600993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:33.606758 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d322ebbc-a465-460a-821b-c8c8b7e1de22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:33.607658 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322ebbc-a465-460a-821b-c8c8b7e1de22 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:33.608138 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322ebbc-a465-460a-821b-c8c8b7e1de22 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:33.608586 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322ebbc-a465-460a-821b-c8c8b7e1de22 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:33.609551 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d322ebbc-a465-460a-821b-c8c8b7e1de22 None admin] RBAC: Adding query filter params (interface=public, service_id=c6a2b92bfd4247b29033fee275f426ea, region_id=RegionOne) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:33.610033 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d322ebbc-a465-460a-821b-c8c8b7e1de22 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:33.612201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d322ebbc-a465-460a-821b-c8c8b7e1de22 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:33.621941 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 504/995] 10.222.0.41 () {56 vars in 1251 bytes} [Fri Jun 3 16:03:33 2022] GET /identity/v3/endpoints?service_id=c6a2b92bfd4247b29033fee275f426ea&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:33.623601 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f63f5-ae64-494a-be4e-b3cac013aa6f None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:33.671842 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:33.727497 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f63f5-ae64-494a-be4e-b3cac013aa6f None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:33.762237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ee5be-c9a0-443a-9ce2-71747842567b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:33.763167 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ee5be-c9a0-443a-9ce2-71747842567b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:33.763754 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ee5be-c9a0-443a-9ce2-71747842567b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:33.764232 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ee5be-c9a0-443a-9ce2-71747842567b None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:33.765080 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ee5be-c9a0-443a-9ce2-71747842567b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:33.766644 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ee5be-c9a0-443a-9ce2-71747842567b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:33.779328 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 492/996] 10.222.0.41 () {56 vars in 1087 bytes} [Fri Jun 3 16:03:33 2022] GET /identity/v3/services => generated 1727 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:03:34.487731 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2b712-6a21-45d6-b0b5-0bf0db072e42 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:34.488235 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2b712-6a21-45d6-b0b5-0bf0db072e42 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:34.488312 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2b712-6a21-45d6-b0b5-0bf0db072e42 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:34.489810 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 505/997] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:34.494985 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab8c66-ff89-4e72-b039-bfb3bd75ecb8 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:34.495285 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab8c66-ff89-4e72-b039-bfb3bd75ecb8 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:34.495605 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab8c66-ff89-4e72-b039-bfb3bd75ecb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:34.512804 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-efab8c66-ff89-4e72-b039-bfb3bd75ecb8 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:34.513929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-efab8c66-ff89-4e72-b039-bfb3bd75ecb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:34.581563 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 493/998] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:34 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:03:34.604303 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2b712-6a21-45d6-b0b5-0bf0db072e42 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:34.707560 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2b712-6a21-45d6-b0b5-0bf0db072e42 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:34.740353 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80108e34-adbd-4b20-b455-57a9b93c852c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:34.741160 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80108e34-adbd-4b20-b455-57a9b93c852c None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:34.741445 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80108e34-adbd-4b20-b455-57a9b93c852c None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:34.741726 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80108e34-adbd-4b20-b455-57a9b93c852c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:34.742373 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80108e34-adbd-4b20-b455-57a9b93c852c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:34.743762 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80108e34-adbd-4b20-b455-57a9b93c852c None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:34.756479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-80108e34-adbd-4b20-b455-57a9b93c852c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 16:03:34.756479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:34.756479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 16:03:34.756479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 16:03:34.756479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 16:03:34.756479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 16:03:34.756479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 16:03:34.756479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:34.756479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 16:03:34.756479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:34.756479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 16:03:34.756479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:34.756479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:34.756479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:34.756479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:34.756479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:34.756479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:34.757445 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:34.757445 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:34.757445 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:34.757445 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:34.757445 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:34.757445 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:34.757445 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 16:03:34.757445 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 16:03:34.757445 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 16:03:34.757445 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 16:03:34.757445 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:34.757445 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:34.757445 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:34.757445 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:34.757445 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:34.757445 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:34.758282 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:34.758282 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:34.758282 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:34.758282 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:34.758282 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:34.758282 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:34.758282 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:34.758282 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:34.758282 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:34.758282 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:34.758282 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 16:03:34.758282 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:34.758282 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 16:03:34.758282 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:34.759704 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 506/999] 10.222.0.41 () {56 vars in 1126 bytes} [Fri Jun 3 16:03:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:34.763755 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab8c66-ff89-4e72-b039-bfb3bd75ecb8 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:34.802537 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:34.862389 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab8c66-ff89-4e72-b039-bfb3bd75ecb8 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:34.896580 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd609f-f3f7-4030-8354-3c40c2b87344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:34.897274 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd609f-f3f7-4030-8354-3c40c2b87344 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:34.897557 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd609f-f3f7-4030-8354-3c40c2b87344 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:34.897835 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd609f-f3f7-4030-8354-3c40c2b87344 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:34.898573 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd609f-f3f7-4030-8354-3c40c2b87344 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:34.898884 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd609f-f3f7-4030-8354-3c40c2b87344 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:34.900353 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd609f-f3f7-4030-8354-3c40c2b87344 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:34.911061 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 494/1000] 10.222.0.41 () {56 vars in 1122 bytes} [Fri Jun 3 16:03:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:34.912662 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80108e34-adbd-4b20-b455-57a9b93c852c None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:34.972781 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:35.016393 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80108e34-adbd-4b20-b455-57a9b93c852c None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:35.049884 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb1401-f1cc-4a89-8946-bf6ee7483fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:35.050555 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb1401-f1cc-4a89-8946-bf6ee7483fe1 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:35.050842 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb1401-f1cc-4a89-8946-bf6ee7483fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:35.051120 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb1401-f1cc-4a89-8946-bf6ee7483fe1 None admin] PATH_INFO: `/v3/services` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:35.051861 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb1401-f1cc-4a89-8946-bf6ee7483fe1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:35.052171 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb1401-f1cc-4a89-8946-bf6ee7483fe1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:35.053560 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb1401-f1cc-4a89-8946-bf6ee7483fe1 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:35.064421 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 507/1001] 10.222.0.41 () {56 vars in 1122 bytes} [Fri Jun 3 16:03:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:35.066265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd609f-f3f7-4030-8354-3c40c2b87344 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:35.165956 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd609f-f3f7-4030-8354-3c40c2b87344 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:35.167491 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:35.200580 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aa33dc-48ce-4955-9677-c6b8fa4963d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:35.201230 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa33dc-48ce-4955-9677-c6b8fa4963d3 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:35.201515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa33dc-48ce-4955-9677-c6b8fa4963d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:35.201793 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa33dc-48ce-4955-9677-c6b8fa4963d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:35.202534 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aa33dc-48ce-4955-9677-c6b8fa4963d3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:35.203927 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aa33dc-48ce-4955-9677-c6b8fa4963d3 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:35.222703 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 495/1002] 10.222.0.41 () {60 vars in 1144 bytes} [Fri Jun 3 16:03:35 2022] POST /identity/v3/endpoints => generated 325 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:03:35.397715 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1484115-8c1a-4d4c-91c1-1ef7cee5a6be None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:35.398050 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1484115-8c1a-4d4c-91c1-1ef7cee5a6be None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:35.398333 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1484115-8c1a-4d4c-91c1-1ef7cee5a6be None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:35.399843 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 508/1003] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:35.405198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6886fbf6-bb4d-424d-875a-2f1ada9e2c5b None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:35.405492 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6886fbf6-bb4d-424d-875a-2f1ada9e2c5b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:35.405769 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6886fbf6-bb4d-424d-875a-2f1ada9e2c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:35.425638 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-6886fbf6-bb4d-424d-875a-2f1ada9e2c5b None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:35.426818 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6886fbf6-bb4d-424d-875a-2f1ada9e2c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:35.501402 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 496/1004] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:35 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:03:35.523903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1484115-8c1a-4d4c-91c1-1ef7cee5a6be None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:35.627556 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1484115-8c1a-4d4c-91c1-1ef7cee5a6be None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:35.650826 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:35.663320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f0926-f537-4446-8968-478a325789ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:35.664151 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f0926-f537-4446-8968-478a325789ad None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:35.664435 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f0926-f537-4446-8968-478a325789ad None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:35.664709 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f0926-f537-4446-8968-478a325789ad None admin] PATH_INFO: `/v3/services/image` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:35.665362 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f0926-f537-4446-8968-478a325789ad None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:35.666706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f0926-f537-4446-8968-478a325789ad None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:35.678870 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-315f0926-f537-4446-8968-478a325789ad None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 03 16:03:35.678870 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:35.678870 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 16:03:35.678870 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 16:03:35.678870 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 16:03:35.678870 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 16:03:35.678870 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 16:03:35.678870 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:35.678870 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 03 16:03:35.678870 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:35.678870 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 16:03:35.678870 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:35.678870 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:35.678870 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:35.678870 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:35.678870 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:35.678870 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:35.679885 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:35.679885 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:35.679885 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:35.679885 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:35.679885 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:35.679885 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:35.679885 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 16:03:35.679885 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 16:03:35.679885 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 16:03:35.679885 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 16:03:35.679885 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:35.679885 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:35.679885 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:35.679885 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:35.679885 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:35.679885 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:35.680719 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:35.680719 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:35.680719 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:35.680719 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:35.680719 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:35.680719 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:35.680719 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:35.680719 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:35.680719 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:35.680719 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:35.680719 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 16:03:35.680719 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:35.680719 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 03 16:03:35.680719 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:35.682247 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 509/1005] 10.222.0.41 () {56 vars in 1105 bytes} [Fri Jun 3 16:03:35 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:35.686076 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6886fbf6-bb4d-424d-875a-2f1ada9e2c5b None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:35.780062 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:35.793372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6886fbf6-bb4d-424d-875a-2f1ada9e2c5b None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:35.828650 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee3239f-5fb0-4154-a0c1-05024fc1e255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:35.829377 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3239f-5fb0-4154-a0c1-05024fc1e255 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:35.829707 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3239f-5fb0-4154-a0c1-05024fc1e255 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:35.829991 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3239f-5fb0-4154-a0c1-05024fc1e255 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:35.830708 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee3239f-5fb0-4154-a0c1-05024fc1e255 None admin] RBAC: Adding query filter params (name=image) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:35.831018 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee3239f-5fb0-4154-a0c1-05024fc1e255 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:35.832431 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee3239f-5fb0-4154-a0c1-05024fc1e255 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:35.837611 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 497/1006] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:35.843148 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f0926-f537-4446-8968-478a325789ad None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:35.943991 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f0926-f537-4446-8968-478a325789ad None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:35.977580 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a00309-84fc-4fe9-9358-b640af38f2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:35.978582 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a00309-84fc-4fe9-9358-b640af38f2e9 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:35.979072 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a00309-84fc-4fe9-9358-b640af38f2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:35.979600 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a00309-84fc-4fe9-9358-b640af38f2e9 None admin] PATH_INFO: `/v3/services` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:35.980520 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a00309-84fc-4fe9-9358-b640af38f2e9 None admin] RBAC: Adding query filter params (type=image) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:35.981052 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a00309-84fc-4fe9-9358-b640af38f2e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:35.982614 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a00309-84fc-4fe9-9358-b640af38f2e9 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:35.987594 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 510/1007] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:35 2022] GET /identity/v3/services?type=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:36.073089 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa800d-bc42-4f0a-b54a-70028f83be90 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:36.073431 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa800d-bc42-4f0a-b54a-70028f83be90 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:36.073646 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa800d-bc42-4f0a-b54a-70028f83be90 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:36.075080 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 498/1008] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:36.081459 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d7f764-eb68-49dd-a81a-5d768d6ba481 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:36.081745 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d7f764-eb68-49dd-a81a-5d768d6ba481 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:36.082010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d7f764-eb68-49dd-a81a-5d768d6ba481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:36.099752 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-d7d7f764-eb68-49dd-a81a-5d768d6ba481 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:36.100966 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-d7d7f764-eb68-49dd-a81a-5d768d6ba481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:36.151903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:36.174448 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 511/1009] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:36 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:03:36.197706 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa800d-bc42-4f0a-b54a-70028f83be90 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:36.308072 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa800d-bc42-4f0a-b54a-70028f83be90 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:36.342551 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ebc9d6-9dd2-4f2c-a516-16bfd07216ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:36.343411 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ebc9d6-9dd2-4f2c-a516-16bfd07216ab None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:36.343702 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ebc9d6-9dd2-4f2c-a516-16bfd07216ab None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:36.343980 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ebc9d6-9dd2-4f2c-a516-16bfd07216ab None admin] PATH_INFO: `/v3/services/compute` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:36.344656 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ebc9d6-9dd2-4f2c-a516-16bfd07216ab None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:36.346091 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ebc9d6-9dd2-4f2c-a516-16bfd07216ab None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:36.357858 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-b5ebc9d6-9dd2-4f2c-a516-16bfd07216ab None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 03 16:03:36.357858 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:36.357858 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 16:03:36.357858 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 16:03:36.357858 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 16:03:36.357858 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 16:03:36.357858 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 16:03:36.357858 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:36.357858 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 03 16:03:36.357858 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:36.357858 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 16:03:36.357858 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:36.357858 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:36.357858 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:36.357858 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:36.357858 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:36.357858 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:36.358959 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:36.358959 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:36.358959 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:36.358959 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:36.358959 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:36.358959 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:36.358959 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 16:03:36.358959 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 16:03:36.358959 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 16:03:36.358959 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 16:03:36.358959 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:36.358959 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:36.358959 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:36.358959 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:36.358959 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:36.358959 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:36.365370 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:36.365370 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:36.365370 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:36.365370 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:36.365370 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:36.365370 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:36.365370 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:36.365370 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:36.365370 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:36.365370 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:36.365370 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 16:03:36.365370 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:36.365370 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 03 16:03:36.365370 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:36.365370 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 499/1010] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:03:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:36.367923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d7f764-eb68-49dd-a81a-5d768d6ba481 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:36.469459 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d7f764-eb68-49dd-a81a-5d768d6ba481 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:36.504401 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967eea07-6714-4c4b-9448-cd99613494bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:36.505068 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967eea07-6714-4c4b-9448-cd99613494bf None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:36.505352 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967eea07-6714-4c4b-9448-cd99613494bf None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:36.505629 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967eea07-6714-4c4b-9448-cd99613494bf None admin] PATH_INFO: `/v3/services` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:36.506331 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967eea07-6714-4c4b-9448-cd99613494bf None admin] RBAC: Adding query filter params (name=compute) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:36.506636 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967eea07-6714-4c4b-9448-cd99613494bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:36.508013 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967eea07-6714-4c4b-9448-cd99613494bf None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:36.517650 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 512/1011] 10.222.0.41 () {56 vars in 1112 bytes} [Fri Jun 3 16:03:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:36.518891 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ebc9d6-9dd2-4f2c-a516-16bfd07216ab None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:36.538419 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:36.620325 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ebc9d6-9dd2-4f2c-a516-16bfd07216ab None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:36.652460 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2eeb05-1fae-4de3-9111-c4186dac02b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:36.653374 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2eeb05-1fae-4de3-9111-c4186dac02b8 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:36.653658 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2eeb05-1fae-4de3-9111-c4186dac02b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:36.653937 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2eeb05-1fae-4de3-9111-c4186dac02b8 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:36.654646 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2eeb05-1fae-4de3-9111-c4186dac02b8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:36.655013 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2eeb05-1fae-4de3-9111-c4186dac02b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:36.656451 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2eeb05-1fae-4de3-9111-c4186dac02b8 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:36.666470 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 500/1012] 10.222.0.41 () {56 vars in 1112 bytes} [Fri Jun 3 16:03:36 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:36.667466 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967eea07-6714-4c4b-9448-cd99613494bf None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:36.714383 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:36.768162 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967eea07-6714-4c4b-9448-cd99613494bf None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:36.803156 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee4653-8aac-4f4a-b841-2540a6af8cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:36.803850 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee4653-8aac-4f4a-b841-2540a6af8cd5 None admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:36.804137 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee4653-8aac-4f4a-b841-2540a6af8cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:36.804416 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee4653-8aac-4f4a-b841-2540a6af8cd5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:36.805155 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aee4653-8aac-4f4a-b841-2540a6af8cd5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:36.806513 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aee4653-8aac-4f4a-b841-2540a6af8cd5 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:36.831337 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 513/1013] 10.222.0.41 () {60 vars in 1144 bytes} [Fri Jun 3 16:03:36 2022] POST /identity/v3/endpoints => generated 334 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:03:37.511544 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7c1b0-8bc6-40f6-be0b-018a45da5cc9 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:37.512036 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7c1b0-8bc6-40f6-be0b-018a45da5cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:37.512036 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7c1b0-8bc6-40f6-be0b-018a45da5cc9 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:37.513534 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 501/1014] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:37.519426 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2c7b4-8f0d-482d-9a86-2feab3634d35 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:37.519726 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2c7b4-8f0d-482d-9a86-2feab3634d35 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:37.520012 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2c7b4-8f0d-482d-9a86-2feab3634d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:37.537538 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-b4c2c7b4-8f0d-482d-9a86-2feab3634d35 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:37.538726 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-b4c2c7b4-8f0d-482d-9a86-2feab3634d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:37.613459 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 514/1015] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:37 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:03:37.636694 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a7c1b0-8bc6-40f6-be0b-018a45da5cc9 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:37.725804 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:37.747223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a7c1b0-8bc6-40f6-be0b-018a45da5cc9 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:37.781824 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783dcfa-fc6a-44c2-9e40-698fed9d659a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:37.783170 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783dcfa-fc6a-44c2-9e40-698fed9d659a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:37.783523 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783dcfa-fc6a-44c2-9e40-698fed9d659a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:37.783763 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783dcfa-fc6a-44c2-9e40-698fed9d659a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:37.784933 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783dcfa-fc6a-44c2-9e40-698fed9d659a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:37.786355 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783dcfa-fc6a-44c2-9e40-698fed9d659a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:37.792620 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 502/1016] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:37.794210 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c2c7b4-8f0d-482d-9a86-2feab3634d35 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:37.822450 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:37.899851 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c2c7b4-8f0d-482d-9a86-2feab3634d35 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:37.936298 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7abac9-1ad2-4149-9b71-74bd6ee7e0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:37.936963 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7abac9-1ad2-4149-9b71-74bd6ee7e0e3 None admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:37.937251 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7abac9-1ad2-4149-9b71-74bd6ee7e0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:37.937530 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7abac9-1ad2-4149-9b71-74bd6ee7e0e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:37.938235 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7abac9-1ad2-4149-9b71-74bd6ee7e0e3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:37.939718 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7abac9-1ad2-4149-9b71-74bd6ee7e0e3 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:37.966405 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 515/1017] 10.222.0.41 () {60 vars in 1141 bytes} [Fri Jun 3 16:03:37 2022] POST /identity/v3/projects => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:03:38.229924 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b65bf-6683-46ff-8c22-4b1c7be99dc5 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:38.230249 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b65bf-6683-46ff-8c22-4b1c7be99dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:38.230500 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b65bf-6683-46ff-8c22-4b1c7be99dc5 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:38.231905 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 503/1018] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:38.237149 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f3564-d041-46e2-80cd-2abf2e78ce4d None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:38.237453 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f3564-d041-46e2-80cd-2abf2e78ce4d None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:38.237738 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f3564-d041-46e2-80cd-2abf2e78ce4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:38.256790 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-fe8f3564-d041-46e2-80cd-2abf2e78ce4d None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:38.257972 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-fe8f3564-d041-46e2-80cd-2abf2e78ce4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:38.327503 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 516/1019] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:38 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:03:38.349577 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b65bf-6683-46ff-8c22-4b1c7be99dc5 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:38.451742 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b65bf-6683-46ff-8c22-4b1c7be99dc5 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:38.485218 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503b603-d594-4ff1-8103-942a2864b4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:38.485801 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503b603-d594-4ff1-8103-942a2864b4fe None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:38.486081 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503b603-d594-4ff1-8103-942a2864b4fe None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:38.486352 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503b603-d594-4ff1-8103-942a2864b4fe None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:38.487070 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a503b603-d594-4ff1-8103-942a2864b4fe None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:38.488464 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a503b603-d594-4ff1-8103-942a2864b4fe None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:38.503029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 504/1020] 10.222.0.41 () {60 vars in 1141 bytes} [Fri Jun 3 16:03:38 2022] POST /identity/v3/services => generated 242 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:03:39.182866 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11c014-5b40-415e-b23f-a554c5cfeb69 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:39.183401 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11c014-5b40-415e-b23f-a554c5cfeb69 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:39.183482 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11c014-5b40-415e-b23f-a554c5cfeb69 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:39.184952 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 517/1021] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:39.190128 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c906dd9-a87e-452c-ab85-66d60a6425d7 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:39.190540 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c906dd9-a87e-452c-ab85-66d60a6425d7 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:39.190726 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c906dd9-a87e-452c-ab85-66d60a6425d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:39.208842 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-0c906dd9-a87e-452c-ab85-66d60a6425d7 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:39.210040 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0c906dd9-a87e-452c-ab85-66d60a6425d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:39.212413 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:39.296167 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 505/1022] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:39 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:03:39.319239 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11c014-5b40-415e-b23f-a554c5cfeb69 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:39.332326 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:39.424395 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11c014-5b40-415e-b23f-a554c5cfeb69 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:39.457294 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b47ca18-dffb-4a24-88e2-a3d989e90d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:39.458040 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47ca18-dffb-4a24-88e2-a3d989e90d73 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:39.458324 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47ca18-dffb-4a24-88e2-a3d989e90d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:39.458600 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47ca18-dffb-4a24-88e2-a3d989e90d73 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:39.468475 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b47ca18-dffb-4a24-88e2-a3d989e90d73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:03:39.477143 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-0b47ca18-dffb-4a24-88e2-a3d989e90d73 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 03 16:03:39.477143 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:39.477143 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:39.477143 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:39.477143 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:39.477143 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:39.477143 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:39.477143 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:39.477143 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:39.477143 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:39.477143 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:39.477143 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:39.477143 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:03:39.477143 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:03:39.477143 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:03:39.477143 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:03:39.478054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:03:39.478054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:03:39.478054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:03:39.478054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:03:39.478054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:39.478054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:39.478054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:39.478054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:39.478054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:39.478054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:39.478054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:39.478054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:39.478054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:39.478054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:39.478054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:39.478054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:39.484087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:39.484087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:39.484087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:39.484087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:39.484087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:03:39.484087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:03:39.484087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:03:39.484087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:03:39.484087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:03:39.484087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:03:39.484087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 03 16:03:39.484087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:39.484087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 518/1023] 10.222.0.41 () {56 vars in 1120 bytes} [Fri Jun 3 16:03:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:39.484805 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c906dd9-a87e-452c-ab85-66d60a6425d7 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:39.583853 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c906dd9-a87e-452c-ab85-66d60a6425d7 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:39.621111 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb61446-29a6-4ea5-854c-ec04602d8b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:39.621781 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb61446-29a6-4ea5-854c-ec04602d8b7a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:39.622064 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb61446-29a6-4ea5-854c-ec04602d8b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:39.622339 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb61446-29a6-4ea5-854c-ec04602d8b7a None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:39.623085 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb61446-29a6-4ea5-854c-ec04602d8b7a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:39.623432 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb61446-29a6-4ea5-854c-ec04602d8b7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:39.624807 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb61446-29a6-4ea5-854c-ec04602d8b7a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:39.626183 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:39.639467 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 506/1024] 10.222.0.41 () {56 vars in 1115 bytes} [Fri Jun 3 16:03:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:39.641013 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b47ca18-dffb-4a24-88e2-a3d989e90d73 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:39.712265 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:39.746767 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b47ca18-dffb-4a24-88e2-a3d989e90d73 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:39.780694 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a3d02-b472-4a34-ba04-3eba331cdbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:39.781486 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a3d02-b472-4a34-ba04-3eba331cdbc1 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:39.781766 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a3d02-b472-4a34-ba04-3eba331cdbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:39.782531 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a3d02-b472-4a34-ba04-3eba331cdbc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:39.793455 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628a3d02-b472-4a34-ba04-3eba331cdbc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:39.794908 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628a3d02-b472-4a34-ba04-3eba331cdbc1 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:39.812539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-628a3d02-b472-4a34-ba04-3eba331cdbc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:39.812539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:39.812539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:39.812539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:39.812539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:39.812539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:39.812539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:39.812539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:39.812539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:39.812539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:39.812539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:39.812539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:39.812539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:03:39.812539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:03:39.812539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:03:39.813436 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:03:39.813436 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:39.813436 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:39.813436 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:39.813436 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:39.813436 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:39.813436 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:39.813436 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:39.813436 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:39.813436 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:39.813436 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:39.813436 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:39.813436 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:39.813436 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:39.813436 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:39.813436 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:39.813436 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:39.814341 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:03:39.814341 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:03:39.814341 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:39.814341 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:39.815378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 519/1025] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:03:39.819424 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb61446-29a6-4ea5-854c-ec04602d8b7a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:39.920049 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb61446-29a6-4ea5-854c-ec04602d8b7a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:39.955131 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b1eb4-1590-4957-b829-d755a40d3bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:39.955867 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b1eb4-1590-4957-b829-d755a40d3bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:39.956156 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b1eb4-1590-4957-b829-d755a40d3bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:39.956432 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b1eb4-1590-4957-b829-d755a40d3bd5 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:39.957177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853b1eb4-1590-4957-b829-d755a40d3bd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:39.957485 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853b1eb4-1590-4957-b829-d755a40d3bd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:39.958897 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853b1eb4-1590-4957-b829-d755a40d3bd5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:39.977306 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 507/1026] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:03:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:39.978846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a3d02-b472-4a34-ba04-3eba331cdbc1 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:40.080031 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a3d02-b472-4a34-ba04-3eba331cdbc1 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:40.102179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:40.114740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd22cd-c81a-4ff9-9e4d-5c75ec97cb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:40.115560 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd22cd-c81a-4ff9-9e4d-5c75ec97cb91 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:40.115842 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd22cd-c81a-4ff9-9e4d-5c75ec97cb91 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:40.116117 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd22cd-c81a-4ff9-9e4d-5c75ec97cb91 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:40.123555 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd22cd-c81a-4ff9-9e4d-5c75ec97cb91 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:40.124969 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd22cd-c81a-4ff9-9e4d-5c75ec97cb91 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:40.138646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-20dd22cd-c81a-4ff9-9e4d-5c75ec97cb91 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 03 16:03:40.138646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:40.138646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:40.138646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:40.138646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:40.138646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:40.138646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:40.138646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:40.138646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:40.138646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:40.138646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:40.138646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:40.138646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:03:40.138646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:03:40.138646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:03:40.138646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:03:40.139627 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:40.139627 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:40.139627 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:03:40.139627 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:03:40.139627 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:03:40.139627 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:03:40.139627 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:40.139627 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:40.139627 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:40.139627 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:40.139627 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:40.139627 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:40.139627 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:40.139627 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:40.139627 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:40.139627 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:40.139627 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:40.139627 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:40.140589 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:40.140589 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:40.140589 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:03:40.140589 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:03:40.140589 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:03:40.140589 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:03:40.140589 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:03:40.140589 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:03:40.140589 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 03 16:03:40.140589 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:40.143610 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 520/1027] 10.222.0.41 () {56 vars in 1093 bytes} [Fri Jun 3 16:03:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:03:40.146008 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b1eb4-1590-4957-b829-d755a40d3bd5 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:40.203544 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:40.255108 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b1eb4-1590-4957-b829-d755a40d3bd5 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:40.283134 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86c228-e58a-4fa2-a47a-13e9c2dbfe50 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:40.283678 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86c228-e58a-4fa2-a47a-13e9c2dbfe50 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:40.283768 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86c228-e58a-4fa2-a47a-13e9c2dbfe50 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:40.285073 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 521/1028] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 16:03:40.289954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c59e1e-8bc1-4842-939e-a26ed45a8507 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:40.290243 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c59e1e-8bc1-4842-939e-a26ed45a8507 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:40.290523 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c59e1e-8bc1-4842-939e-a26ed45a8507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:40.291996 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e037fb-17f0-4777-959f-c2db4348baec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:40.292659 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e037fb-17f0-4777-959f-c2db4348baec None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:40.292943 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e037fb-17f0-4777-959f-c2db4348baec None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:40.293214 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e037fb-17f0-4777-959f-c2db4348baec None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:40.294043 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e037fb-17f0-4777-959f-c2db4348baec None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:40.294378 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e037fb-17f0-4777-959f-c2db4348baec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:40.295806 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e037fb-17f0-4777-959f-c2db4348baec None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:40.307396 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-20c59e1e-8bc1-4842-939e-a26ed45a8507 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:40.308600 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-20c59e1e-8bc1-4842-939e-a26ed45a8507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:40.315363 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 508/1029] 10.222.0.41 () {56 vars in 1133 bytes} [Fri Jun 3 16:03:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:40.320735 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e037fb-17f0-4777-959f-c2db4348baec None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:40.388079 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 522/1030] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 16:03:40.413327 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c59e1e-8bc1-4842-939e-a26ed45a8507 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:40.432157 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e037fb-17f0-4777-959f-c2db4348baec None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:40.449613 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:40.474721 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476387ad-dcb1-4634-9190-eae093ee8a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:40.475503 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476387ad-dcb1-4634-9190-eae093ee8a30 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:40.475793 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476387ad-dcb1-4634-9190-eae093ee8a30 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:40.476319 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476387ad-dcb1-4634-9190-eae093ee8a30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:40.518961 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c59e1e-8bc1-4842-939e-a26ed45a8507 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:40.533224 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:40.549835 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476387ad-dcb1-4634-9190-eae093ee8a30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:40.551337 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476387ad-dcb1-4634-9190-eae093ee8a30 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:40.555631 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b430087-5c20-43c0-bbeb-dddae12f9ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:40.556436 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b430087-5c20-43c0-bbeb-dddae12f9ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:40.556718 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b430087-5c20-43c0-bbeb-dddae12f9ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:40.556991 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b430087-5c20-43c0-bbeb-dddae12f9ea1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:40.558150 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b430087-5c20-43c0-bbeb-dddae12f9ea1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:40.559593 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b430087-5c20-43c0-bbeb-dddae12f9ea1 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:40.561273 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 523/1031] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:40.567225 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b430087-5c20-43c0-bbeb-dddae12f9ea1 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:40.655327 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 509/1032] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:03:40 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 03 16:03:40.659495 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476387ad-dcb1-4634-9190-eae093ee8a30 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:40.674975 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:40.675689 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b430087-5c20-43c0-bbeb-dddae12f9ea1 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:40.717173 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668e2e4-74b7-42a0-8c97-59726db3929a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:40.717843 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668e2e4-74b7-42a0-8c97-59726db3929a None admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:40.718125 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668e2e4-74b7-42a0-8c97-59726db3929a None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:40.718820 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668e2e4-74b7-42a0-8c97-59726db3929a None admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:40.719572 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a668e2e4-74b7-42a0-8c97-59726db3929a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:40.721127 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a668e2e4-74b7-42a0-8c97-59726db3929a None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:40.741981 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 524/1033] 10.222.0.41 () {60 vars in 1132 bytes} [Fri Jun 3 16:03:40 2022] POST /identity/v3/users => generated 285 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:03:40.780813 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476387ad-dcb1-4634-9190-eae093ee8a30 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:40.790574 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73d0dd-7f99-4a82-aa5e-e192791d3617 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:40.790930 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73d0dd-7f99-4a82-aa5e-e192791d3617 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:40.791150 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73d0dd-7f99-4a82-aa5e-e192791d3617 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:40.792769 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 525/1034] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 16:03:40.798767 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac9670-84ae-4014-a4b1-7aba334c9428 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:40.799117 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac9670-84ae-4014-a4b1-7aba334c9428 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:40.799370 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac9670-84ae-4014-a4b1-7aba334c9428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:40.818089 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-dcac9670-84ae-4014-a4b1-7aba334c9428 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:40.819361 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-dcac9670-84ae-4014-a4b1-7aba334c9428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:40.824526 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4348cd-937e-47f7-8588-cb3a05e952b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:40.825343 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4348cd-937e-47f7-8588-cb3a05e952b2 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:40.825626 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4348cd-937e-47f7-8588-cb3a05e952b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:40.825901 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4348cd-937e-47f7-8588-cb3a05e952b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:40.838879 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4348cd-937e-47f7-8588-cb3a05e952b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:40.840314 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4348cd-937e-47f7-8588-cb3a05e952b2 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:40.856961 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:40.859215 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-ff4348cd-937e-47f7-8588-cb3a05e952b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:40.859215 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:40.859215 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:40.859215 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:40.859215 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:40.859215 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:40.859215 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:40.859215 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:40.859215 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:40.859215 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:40.859215 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:40.859215 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:40.859215 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:03:40.859215 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:03:40.859215 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:03:40.860345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:03:40.860345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:40.860345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:40.860345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:40.860345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:40.860345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:40.860345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:40.860345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:40.860345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:40.860345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:40.860345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:40.860345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:40.860345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:40.860345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:40.860345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:40.860345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:40.860345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:40.861246 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:03:40.861246 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:03:40.861246 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:40.861246 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:40.861246 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 510/1035] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:03:40.866396 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4348cd-937e-47f7-8588-cb3a05e952b2 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:40.909277 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 526/1036] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 16:03:40.933021 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac9670-84ae-4014-a4b1-7aba334c9428 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:40.946688 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:40.986867 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4348cd-937e-47f7-8588-cb3a05e952b2 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:41.025791 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a66e2d0-92ea-43b5-a6aa-af5a41ff2433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:41.026453 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a66e2d0-92ea-43b5-a6aa-af5a41ff2433 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:41.026736 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a66e2d0-92ea-43b5-a6aa-af5a41ff2433 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:41.027014 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a66e2d0-92ea-43b5-a6aa-af5a41ff2433 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:41.028111 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a66e2d0-92ea-43b5-a6aa-af5a41ff2433 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:41.028419 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a66e2d0-92ea-43b5-a6aa-af5a41ff2433 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:41.029783 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a66e2d0-92ea-43b5-a6aa-af5a41ff2433 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:41.041719 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 511/1037] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:41.047103 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a66e2d0-92ea-43b5-a6aa-af5a41ff2433 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:41.047400 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac9670-84ae-4014-a4b1-7aba334c9428 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:41.084764 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea689c7d-cfad-4a41-97b3-59d549512abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:41.085798 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea689c7d-cfad-4a41-97b3-59d549512abe None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:41.086270 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea689c7d-cfad-4a41-97b3-59d549512abe None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:41.086552 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea689c7d-cfad-4a41-97b3-59d549512abe None admin] PATH_INFO: `/v3/services/image` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:41.087204 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea689c7d-cfad-4a41-97b3-59d549512abe None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:41.088599 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea689c7d-cfad-4a41-97b3-59d549512abe None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:41.100811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-ea689c7d-cfad-4a41-97b3-59d549512abe None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 03 16:03:41.100811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:41.100811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 16:03:41.100811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 16:03:41.100811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 16:03:41.100811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 16:03:41.100811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 16:03:41.100811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:41.100811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 03 16:03:41.100811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:41.100811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 16:03:41.100811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:41.100811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:41.100811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:41.100811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:41.100811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:41.100811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:41.101807 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:41.101807 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:41.101807 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:41.101807 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:41.101807 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:41.101807 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:41.101807 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 16:03:41.101807 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 16:03:41.101807 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 16:03:41.101807 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 16:03:41.101807 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:41.101807 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:41.101807 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:41.101807 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:41.101807 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:41.101807 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:41.102688 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:41.102688 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:41.102688 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:41.102688 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:41.102688 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:41.102688 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:41.102688 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:41.102688 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:41.102688 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:41.102688 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:41.102688 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 16:03:41.102688 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:41.102688 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 03 16:03:41.102688 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:41.102688 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 527/1038] 10.222.0.41 () {56 vars in 1105 bytes} [Fri Jun 3 16:03:40 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:03:41.108203 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea689c7d-cfad-4a41-97b3-59d549512abe None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:41.140713 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:41.156071 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a66e2d0-92ea-43b5-a6aa-af5a41ff2433 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:41.183240 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:41.193117 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690dd8f8-573f-421f-bcef-003760661187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:41.193935 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690dd8f8-573f-421f-bcef-003760661187 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:41.194213 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690dd8f8-573f-421f-bcef-003760661187 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:41.194495 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690dd8f8-573f-421f-bcef-003760661187 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:41.201724 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690dd8f8-573f-421f-bcef-003760661187 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:41.203180 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690dd8f8-573f-421f-bcef-003760661187 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:41.215784 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea689c7d-cfad-4a41-97b3-59d549512abe None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:41.217377 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-690dd8f8-573f-421f-bcef-003760661187 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:03:41.217377 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:41.217377 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:41.217377 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:41.217377 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:41.217377 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:41.217377 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:41.217377 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:41.217377 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:41.217377 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:41.217377 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:41.217377 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:41.217377 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:03:41.217377 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:03:41.217377 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:03:41.218564 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:03:41.218564 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:41.218564 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:41.218564 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:41.218564 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:41.218564 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:41.218564 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:41.218564 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:41.218564 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:41.218564 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:41.218564 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:41.218564 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:41.218564 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:41.218564 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:41.218564 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:41.218564 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:41.218564 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:41.220219 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:03:41.220219 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:03:41.220219 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:03:41.220219 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:03:41.220219 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:03:41.220219 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:03:41.220219 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:03:41.220219 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:41.220219 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 512/1039] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:03:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:03:41.226348 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690dd8f8-573f-421f-bcef-003760661187 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:41.254822 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef398d10-35bd-4c9a-b165-05f4ed5f3597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:41.255539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef398d10-35bd-4c9a-b165-05f4ed5f3597 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:41.255830 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef398d10-35bd-4c9a-b165-05f4ed5f3597 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:41.256110 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef398d10-35bd-4c9a-b165-05f4ed5f3597 None admin] PATH_INFO: `/v3/services` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:41.256822 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef398d10-35bd-4c9a-b165-05f4ed5f3597 None admin] RBAC: Adding query filter params (name=image) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:41.257137 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef398d10-35bd-4c9a-b165-05f4ed5f3597 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:41.258493 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef398d10-35bd-4c9a-b165-05f4ed5f3597 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:41.268271 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 528/1040] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:41.268956 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef398d10-35bd-4c9a-b165-05f4ed5f3597 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:41.335939 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690dd8f8-573f-421f-bcef-003760661187 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:41.375238 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a7032f-e175-4c37-b5e9-fac615e676f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:41.376506 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7032f-e175-4c37-b5e9-fac615e676f6 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:41.376698 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:41.377134 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7032f-e175-4c37-b5e9-fac615e676f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:41.377134 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7032f-e175-4c37-b5e9-fac615e676f6 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:41.377921 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a7032f-e175-4c37-b5e9-fac615e676f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:41.378237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a7032f-e175-4c37-b5e9-fac615e676f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:41.379727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a7032f-e175-4c37-b5e9-fac615e676f6 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:41.385134 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef398d10-35bd-4c9a-b165-05f4ed5f3597 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:41.393630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 513/1041] 10.222.0.41 () {56 vars in 1148 bytes} [Fri Jun 3 16:03:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:41.399457 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a7032f-e175-4c37-b5e9-fac615e676f6 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:41.426400 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d039674-c7ef-4552-ac11-5d4bef110f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:41.427063 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d039674-c7ef-4552-ac11-5d4bef110f8f None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:41.427383 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d039674-c7ef-4552-ac11-5d4bef110f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:41.427663 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d039674-c7ef-4552-ac11-5d4bef110f8f None admin] PATH_INFO: `/v3/services` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:41.428370 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d039674-c7ef-4552-ac11-5d4bef110f8f None admin] RBAC: Adding query filter params (type=image) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:41.428690 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d039674-c7ef-4552-ac11-5d4bef110f8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:41.430032 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d039674-c7ef-4552-ac11-5d4bef110f8f None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:41.435050 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 529/1042] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:41 2022] GET /identity/v3/services?type=image => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:41.441214 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d039674-c7ef-4552-ac11-5d4bef110f8f None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:41.452949 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:41.510116 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a7032f-e175-4c37-b5e9-fac615e676f6 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:41.546204 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837bc07-42d2-4c03-b5cc-3ca54c49f885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:41.546856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837bc07-42d2-4c03-b5cc-3ca54c49f885 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:41.547139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837bc07-42d2-4c03-b5cc-3ca54c49f885 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:41.548095 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837bc07-42d2-4c03-b5cc-3ca54c49f885 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:41.548931 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8837bc07-42d2-4c03-b5cc-3ca54c49f885 None admin] RBAC: Adding query filter params (role.id=3552bc334c464cd19e06a671ad16af6c, scope.project.id=c274254e510a468e9acbecc6298301cc, user.id=2fd08ea39ee94b92ac6763608e69f158) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:41.549245 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8837bc07-42d2-4c03-b5cc-3ca54c49f885 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:41.549520 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d039674-c7ef-4552-ac11-5d4bef110f8f None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:41.550680 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8837bc07-42d2-4c03-b5cc-3ca54c49f885 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:41.563765 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 514/1043] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:03:41 2022] GET /identity/v3/role_assignments?user.id=2fd08ea39ee94b92ac6763608e69f158&scope.project.id=c274254e510a468e9acbecc6298301cc&role.id=3552bc334c464cd19e06a671ad16af6c => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:41.602093 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29001800-aac9-4b7c-a9e1-00290b3c6bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:41.602688 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29001800-aac9-4b7c-a9e1-00290b3c6bec None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:41.602975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29001800-aac9-4b7c-a9e1-00290b3c6bec None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:41.603254 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29001800-aac9-4b7c-a9e1-00290b3c6bec None admin] PATH_INFO: `/v3/endpoints` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:41.604104 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29001800-aac9-4b7c-a9e1-00290b3c6bec None admin] RBAC: Adding query filter params (interface=public, service_id=a7cd88b7befd462fb030bf8dfdb3daf5, region_id=RegionOne) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:41.604420 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29001800-aac9-4b7c-a9e1-00290b3c6bec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:41.605793 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29001800-aac9-4b7c-a9e1-00290b3c6bec None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:41.612490 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 530/1044] 10.222.0.41 () {56 vars in 1251 bytes} [Fri Jun 3 16:03:41 2022] GET /identity/v3/endpoints?service_id=a7cd88b7befd462fb030bf8dfdb3daf5&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:41.617775 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837bc07-42d2-4c03-b5cc-3ca54c49f885 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:41.727085 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837bc07-42d2-4c03-b5cc-3ca54c49f885 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:41.738834 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:41.765850 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537ff9b-f31c-457c-adf1-f7925e3b88c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:41.766543 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537ff9b-f31c-457c-adf1-f7925e3b88c3 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:41.766831 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537ff9b-f31c-457c-adf1-f7925e3b88c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:41.767106 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537ff9b-f31c-457c-adf1-f7925e3b88c3 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:41.767844 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7537ff9b-f31c-457c-adf1-f7925e3b88c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:41.769203 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7537ff9b-f31c-457c-adf1-f7925e3b88c3 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:41.776391 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 515/1045] 10.222.0.41 () {56 vars in 1087 bytes} [Fri Jun 3 16:03:41 2022] GET /identity/v3/services => generated 1957 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:03:43.050156 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9a51d-9a35-4a7c-ab0d-ab1114f10189 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:43.050701 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9a51d-9a35-4a7c-ab0d-ab1114f10189 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:43.050701 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9a51d-9a35-4a7c-ab0d-ab1114f10189 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:43.052262 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 531/1046] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:43.057202 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3578c43-652c-4b5a-b585-56777993de5d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:43.057611 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3578c43-652c-4b5a-b585-56777993de5d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:43.057783 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3578c43-652c-4b5a-b585-56777993de5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:43.074196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c3578c43-652c-4b5a-b585-56777993de5d None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:43.075394 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c3578c43-652c-4b5a-b585-56777993de5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:43.142873 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 516/1047] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:03:43.166764 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9a51d-9a35-4a7c-ab0d-ab1114f10189 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:43.191962 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:43.275592 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9a51d-9a35-4a7c-ab0d-ab1114f10189 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:43.311101 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41fe695-c985-4d93-9e19-7f86f5108c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:43.311848 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41fe695-c985-4d93-9e19-7f86f5108c6b None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:43.312135 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41fe695-c985-4d93-9e19-7f86f5108c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:43.312437 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41fe695-c985-4d93-9e19-7f86f5108c6b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:43.322697 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41fe695-c985-4d93-9e19-7f86f5108c6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:03:43.330903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-d41fe695-c985-4d93-9e19-7f86f5108c6b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:03:43.330903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:43.330903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:43.330903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:43.330903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:43.330903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:43.330903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:43.330903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:43.330903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:43.330903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:43.330903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:43.330903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:43.330903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:03:43.330903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:03:43.330903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:03:43.330903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:03:43.332024 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:03:43.332024 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:03:43.332024 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:03:43.332024 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:03:43.332024 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:43.332024 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:43.332024 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:43.332024 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:43.332024 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:43.332024 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:43.332024 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:43.332024 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:43.332024 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:43.332024 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:43.332024 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:43.332024 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:43.332850 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:43.332850 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:43.332850 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:43.332850 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:43.332850 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:03:43.332850 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:03:43.332850 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:03:43.332850 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:03:43.332850 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:03:43.332850 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:03:43.332850 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:03:43.332850 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:43.332850 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 532/1048] 10.222.0.41 () {56 vars in 1096 bytes} [Fri Jun 3 16:03:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:43.339533 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3578c43-652c-4b5a-b585-56777993de5d None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:43.439787 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3578c43-652c-4b5a-b585-56777993de5d None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:43.442109 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:43.478019 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee01b6-6d1f-41cb-96be-ff1cdf38ea36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:43.478710 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee01b6-6d1f-41cb-96be-ff1cdf38ea36 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:43.479002 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee01b6-6d1f-41cb-96be-ff1cdf38ea36 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:43.479279 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee01b6-6d1f-41cb-96be-ff1cdf38ea36 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:43.480081 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ee01b6-6d1f-41cb-96be-ff1cdf38ea36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:43.480392 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ee01b6-6d1f-41cb-96be-ff1cdf38ea36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:43.481810 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ee01b6-6d1f-41cb-96be-ff1cdf38ea36 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:43.495593 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 517/1049] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:03:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:43.497044 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41fe695-c985-4d93-9e19-7f86f5108c6b None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:43.581206 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:43.600545 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41fe695-c985-4d93-9e19-7f86f5108c6b None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:43.640150 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087cfedd-359c-48fe-8a0b-a500a20f03af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:43.641025 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087cfedd-359c-48fe-8a0b-a500a20f03af None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:43.641868 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087cfedd-359c-48fe-8a0b-a500a20f03af None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:43.642147 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087cfedd-359c-48fe-8a0b-a500a20f03af None admin] PATH_INFO: `/v3/users/ff5049ddc4244e50b65cd0e5bf221639` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:43.655843 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087cfedd-359c-48fe-8a0b-a500a20f03af None admin] RBAC: Authorizing `identity:get_user(user_id=ff5049ddc4244e50b65cd0e5bf221639)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:43.657364 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087cfedd-359c-48fe-8a0b-a500a20f03af None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:43.663641 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 533/1050] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:03:43 2022] GET /identity/v3/users/ff5049ddc4244e50b65cd0e5bf221639 => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:43.664925 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee01b6-6d1f-41cb-96be-ff1cdf38ea36 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:43.771764 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee01b6-6d1f-41cb-96be-ff1cdf38ea36 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:43.802360 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9b2ad-f5ea-4e8b-9dcf-db29e00d56e3 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:43.802642 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9b2ad-f5ea-4e8b-9dcf-db29e00d56e3 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:43.802921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9b2ad-f5ea-4e8b-9dcf-db29e00d56e3 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:43.804471 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 534/1051] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 16:03:43.806744 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ca093-c95b-4709-b7d0-4090580ea6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:43.807417 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ca093-c95b-4709-b7d0-4090580ea6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:43.807707 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ca093-c95b-4709-b7d0-4090580ea6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:43.807965 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ca093-c95b-4709-b7d0-4090580ea6c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:43.809156 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfaeac0-69ba-4dc9-ba51-f09088990029 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:43.809450 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfaeac0-69ba-4dc9-ba51-f09088990029 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:43.809722 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfaeac0-69ba-4dc9-ba51-f09088990029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:43.826670 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-fcfaeac0-69ba-4dc9-ba51-f09088990029 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:43.827888 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-fcfaeac0-69ba-4dc9-ba51-f09088990029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:43.845306 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:43.882129 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ca093-c95b-4709-b7d0-4090580ea6c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:43.883604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ca093-c95b-4709-b7d0-4090580ea6c0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:43.899102 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 535/1052] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:03:43.922099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfaeac0-69ba-4dc9-ba51-f09088990029 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:43.994333 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 518/1053] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:03:43 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 16:03:43.995727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ca093-c95b-4709-b7d0-4090580ea6c0 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:44.001587 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:44.037193 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfaeac0-69ba-4dc9-ba51-f09088990029 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:44.062914 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:44.071877 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe04441-5955-41b9-83f2-ebce70476da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:44.072691 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe04441-5955-41b9-83f2-ebce70476da2 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:44.072976 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe04441-5955-41b9-83f2-ebce70476da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:44.073254 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe04441-5955-41b9-83f2-ebce70476da2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:44.083170 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe04441-5955-41b9-83f2-ebce70476da2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:03:44.091320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-bfe04441-5955-41b9-83f2-ebce70476da2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 03 16:03:44.091320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:44.091320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:44.091320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:44.091320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:44.091320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:44.091320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:44.091320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:44.091320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:44.091320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:44.091320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:44.091320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:44.091320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:03:44.091320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:03:44.091320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:03:44.091320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:03:44.092251 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:03:44.092251 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:03:44.092251 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:03:44.092251 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:03:44.092251 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:44.092251 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:44.092251 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:44.092251 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:44.092251 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:44.092251 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:44.092251 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:44.092251 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:44.092251 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:44.092251 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:44.092251 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:44.092251 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:44.093100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:44.093100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:44.093100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:44.093100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:44.093100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:03:44.093100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:03:44.093100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:03:44.093100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:03:44.093100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:03:44.093100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:03:44.093100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 03 16:03:44.093100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:44.093100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 536/1054] 10.222.0.41 () {56 vars in 1120 bytes} [Fri Jun 3 16:03:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:03:44.093888 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbaabb-755f-4d68-9de9-cf40af1ab0b7 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:44.093956 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbaabb-755f-4d68-9de9-cf40af1ab0b7 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:44.094735 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbaabb-755f-4d68-9de9-cf40af1ab0b7 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:44.096499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 537/1055] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:44.098796 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bbaabb-755f-4d68-9de9-cf40af1ab0b7 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:44.099273 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ca093-c95b-4709-b7d0-4090580ea6c0 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:44.132879 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3bea11-fd14-4a43-96c6-f625b7074e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:44.133731 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bea11-fd14-4a43-96c6-f625b7074e46 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:44.134015 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bea11-fd14-4a43-96c6-f625b7074e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:44.134289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bea11-fd14-4a43-96c6-f625b7074e46 None admin] PATH_INFO: `/v3/projects/24b01e54c67341d490411f88e85a39f1` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:44.135499 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3bea11-fd14-4a43-96c6-f625b7074e46 None admin] RBAC: Authorizing `identity:get_project(project_id=24b01e54c67341d490411f88e85a39f1)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:44.136936 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3bea11-fd14-4a43-96c6-f625b7074e46 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:44.138631 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 519/1056] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:03:43 2022] GET /identity/v3/projects/24b01e54c67341d490411f88e85a39f1 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:44.139792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10570b-4a21-4730-8fe4-046857243254 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:44.140085 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10570b-4a21-4730-8fe4-046857243254 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:44.140355 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10570b-4a21-4730-8fe4-046857243254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:44.156274 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-4e10570b-4a21-4730-8fe4-046857243254 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:44.157922 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4e10570b-4a21-4730-8fe4-046857243254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:44.199693 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bbaabb-755f-4d68-9de9-cf40af1ab0b7 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:44.221254 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 520/1057] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:03:44.222056 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10570b-4a21-4730-8fe4-046857243254 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:44.234037 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:44.242509 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec89922-e74f-43ef-a8e7-0e2bce095363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:44.243224 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec89922-e74f-43ef-a8e7-0e2bce095363 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:44.243565 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec89922-e74f-43ef-a8e7-0e2bce095363 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:44.243845 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec89922-e74f-43ef-a8e7-0e2bce095363 None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:44.244803 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec89922-e74f-43ef-a8e7-0e2bce095363 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:44.245117 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec89922-e74f-43ef-a8e7-0e2bce095363 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:44.249388 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec89922-e74f-43ef-a8e7-0e2bce095363 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:44.257884 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 538/1058] 10.222.0.41 () {56 vars in 1115 bytes} [Fri Jun 3 16:03:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:44.258681 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec89922-e74f-43ef-a8e7-0e2bce095363 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:44.290978 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:44.332293 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10570b-4a21-4730-8fe4-046857243254 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:44.356536 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec89922-e74f-43ef-a8e7-0e2bce095363 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:44.367039 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2acd3de-41bc-4744-acdf-c86dc5fa06bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:44.367726 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2acd3de-41bc-4744-acdf-c86dc5fa06bc None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:44.368011 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2acd3de-41bc-4744-acdf-c86dc5fa06bc None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:44.368288 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2acd3de-41bc-4744-acdf-c86dc5fa06bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:44.369123 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2acd3de-41bc-4744-acdf-c86dc5fa06bc None admin] RBAC: Adding query filter params (role.id=75936a608b8149099ff54e317f3a6740, scope.project.id=24b01e54c67341d490411f88e85a39f1, user.id=ff5049ddc4244e50b65cd0e5bf221639) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:44.369434 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2acd3de-41bc-4744-acdf-c86dc5fa06bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:44.370866 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2acd3de-41bc-4744-acdf-c86dc5fa06bc None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:44.376256 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 521/1059] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:03:44 2022] GET /identity/v3/role_assignments?user.id=ff5049ddc4244e50b65cd0e5bf221639&scope.project.id=24b01e54c67341d490411f88e85a39f1&role.id=75936a608b8149099ff54e317f3a6740 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:44.377021 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2acd3de-41bc-4744-acdf-c86dc5fa06bc None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:44.389645 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c1702-be32-44b6-b573-e2ba9640466c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:44.390556 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c1702-be32-44b6-b573-e2ba9640466c None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:44.390843 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c1702-be32-44b6-b573-e2ba9640466c None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:44.391153 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c1702-be32-44b6-b573-e2ba9640466c None admin] PATH_INFO: `/v3/services/image` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:44.391935 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c1702-be32-44b6-b573-e2ba9640466c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:44.393352 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c1702-be32-44b6-b573-e2ba9640466c None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:44.405921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-7f4c1702-be32-44b6-b573-e2ba9640466c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 03 16:03:44.405921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:44.405921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 16:03:44.405921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 16:03:44.405921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 16:03:44.405921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 16:03:44.405921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 16:03:44.405921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:44.405921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 03 16:03:44.405921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:44.405921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 16:03:44.405921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:44.405921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:44.405921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:44.405921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:44.405921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:44.405921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:44.407586 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:44.407586 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:44.407586 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:44.407586 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:44.407586 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:44.407586 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:44.407586 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 16:03:44.407586 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 16:03:44.407586 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 16:03:44.407586 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 16:03:44.407586 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:44.407586 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:44.407586 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:44.407586 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:44.407586 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:44.407586 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:44.408441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:44.408441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:44.408441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:44.408441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:44.408441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:44.408441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:44.408441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:44.408441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:44.408441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:44.408441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:44.408441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 16:03:44.408441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:03:44.408441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 03 16:03:44.408441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:44.408441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 539/1060] 10.222.0.41 () {56 vars in 1105 bytes} [Fri Jun 3 16:03:44 2022] GET /identity/v3/services/image => generated 86 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:44.413968 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c1702-be32-44b6-b573-e2ba9640466c None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:44.476809 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:44.491373 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2acd3de-41bc-4744-acdf-c86dc5fa06bc None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:44.520601 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:44.527871 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c1702-be32-44b6-b573-e2ba9640466c None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:44.530314 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517d275a-3d25-44a4-bdac-bff4e144814c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:44.531125 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d275a-3d25-44a4-bdac-bff4e144814c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:44.531444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d275a-3d25-44a4-bdac-bff4e144814c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:44.531757 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d275a-3d25-44a4-bdac-bff4e144814c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:44.543184 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517d275a-3d25-44a4-bdac-bff4e144814c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:44.544854 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517d275a-3d25-44a4-bdac-bff4e144814c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:44.564303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-517d275a-3d25-44a4-bdac-bff4e144814c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:44.564303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:44.564303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:44.564303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:44.564303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:44.564303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:44.564303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:44.564303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:44.564303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:44.564303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:44.564303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:44.564303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:44.564303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:03:44.564303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:03:44.564303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:03:44.565270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:03:44.565270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:44.565270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:44.565270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:44.565270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:44.565270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:44.565270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:44.565270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:44.565270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:44.565270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:44.565270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:44.565270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:44.565270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:44.565270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:44.565270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:44.565270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:44.565270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:44.566176 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:03:44.566176 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:03:44.566176 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:44.566176 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:44.566399 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b18e6-fec2-43a5-9b2b-abf65ffbf6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:44.566399 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b18e6-fec2-43a5-9b2b-abf65ffbf6d1 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:44.566399 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b18e6-fec2-43a5-9b2b-abf65ffbf6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:44.566612 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 522/1061] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:44.566681 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b18e6-fec2-43a5-9b2b-abf65ffbf6d1 None admin] PATH_INFO: `/v3/services` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:44.567278 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b18e6-fec2-43a5-9b2b-abf65ffbf6d1 None admin] RBAC: Adding query filter params (name=image) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:44.567278 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b18e6-fec2-43a5-9b2b-abf65ffbf6d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:44.568497 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b18e6-fec2-43a5-9b2b-abf65ffbf6d1 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:44.571789 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517d275a-3d25-44a4-bdac-bff4e144814c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:44.573836 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 540/1062] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:44 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:44.579242 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b18e6-fec2-43a5-9b2b-abf65ffbf6d1 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:44.679745 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517d275a-3d25-44a4-bdac-bff4e144814c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:44.690409 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b18e6-fec2-43a5-9b2b-abf65ffbf6d1 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:44.711043 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:44.724031 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e992da53-12b7-446c-9746-74e3d46922be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:44.724986 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e992da53-12b7-446c-9746-74e3d46922be None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:44.725493 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e992da53-12b7-446c-9746-74e3d46922be None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:44.725978 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e992da53-12b7-446c-9746-74e3d46922be None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:44.726445 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b2304-776c-450e-93ec-ea6629c1962b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:44.727109 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b2304-776c-450e-93ec-ea6629c1962b None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:44.727380 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e992da53-12b7-446c-9746-74e3d46922be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:44.727499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b2304-776c-450e-93ec-ea6629c1962b None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:44.727693 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b2304-776c-450e-93ec-ea6629c1962b None admin] PATH_INFO: `/v3/services` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:44.728185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e992da53-12b7-446c-9746-74e3d46922be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:44.728402 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b2304-776c-450e-93ec-ea6629c1962b None admin] RBAC: Adding query filter params (type=image) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:44.728710 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b2304-776c-450e-93ec-ea6629c1962b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:44.730062 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b2304-776c-450e-93ec-ea6629c1962b None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:44.730371 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e992da53-12b7-446c-9746-74e3d46922be None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:44.735582 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 541/1063] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:44 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:44.741024 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b2304-776c-450e-93ec-ea6629c1962b None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:44.749574 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 523/1064] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:03:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:44.751365 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e992da53-12b7-446c-9746-74e3d46922be None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:44.792587 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:44.848802 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b2304-776c-450e-93ec-ea6629c1962b None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:44.853743 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e992da53-12b7-446c-9746-74e3d46922be None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:44.885820 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1701dada-ffcd-417a-83b4-cc69cd9321d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:44.886484 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701dada-ffcd-417a-83b4-cc69cd9321d6 None admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:44.886769 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701dada-ffcd-417a-83b4-cc69cd9321d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:44.887048 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701dada-ffcd-417a-83b4-cc69cd9321d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:44.887813 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1701dada-ffcd-417a-83b4-cc69cd9321d6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:44.889173 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1701dada-ffcd-417a-83b4-cc69cd9321d6 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:44.889709 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d950fa9f-1235-47c7-a397-80e37d28a53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:44.890517 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950fa9f-1235-47c7-a397-80e37d28a53c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:44.890803 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950fa9f-1235-47c7-a397-80e37d28a53c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:44.891712 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950fa9f-1235-47c7-a397-80e37d28a53c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:44.899232 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d950fa9f-1235-47c7-a397-80e37d28a53c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:44.900779 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d950fa9f-1235-47c7-a397-80e37d28a53c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:44.919427 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 542/1065] 10.222.0.41 () {60 vars in 1144 bytes} [Fri Jun 3 16:03:44 2022] POST /identity/v3/endpoints => generated 327 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:03:44.919630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-d950fa9f-1235-47c7-a397-80e37d28a53c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 03 16:03:44.919630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:44.919630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:44.919630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:44.919630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:44.919630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:44.919630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:44.919630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:44.919630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:44.919630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:44.919630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:44.919630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:44.919630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:03:44.919630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:03:44.919630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:03:44.919630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:03:44.920613 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:44.920613 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:44.920613 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:03:44.920613 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:03:44.920613 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:03:44.920613 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:03:44.920613 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:44.920613 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:44.920613 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:44.920613 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:44.920613 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:44.920613 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:44.920613 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:44.920613 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:44.920613 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:44.920613 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:44.920613 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:44.920613 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:44.921556 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:44.921556 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:44.921556 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:03:44.921556 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:03:44.921556 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:03:44.921556 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:03:44.921556 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:03:44.921556 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:03:44.921556 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 03 16:03:44.921556 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:44.921556 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 524/1066] 10.222.0.41 () {56 vars in 1093 bytes} [Fri Jun 3 16:03:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:03:44.922989 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1701dada-ffcd-417a-83b4-cc69cd9321d6 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:44.973536 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:45.046417 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1701dada-ffcd-417a-83b4-cc69cd9321d6 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:45.083043 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822be948-0343-4739-b620-7a98006f5c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:45.083889 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822be948-0343-4739-b620-7a98006f5c6d None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:45.084055 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822be948-0343-4739-b620-7a98006f5c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:45.084317 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822be948-0343-4739-b620-7a98006f5c6d None admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:45.085174 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822be948-0343-4739-b620-7a98006f5c6d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:45.085489 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822be948-0343-4739-b620-7a98006f5c6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:45.086935 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822be948-0343-4739-b620-7a98006f5c6d None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:45.110364 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 543/1067] 10.222.0.41 () {56 vars in 1133 bytes} [Fri Jun 3 16:03:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:45.112057 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d950fa9f-1235-47c7-a397-80e37d28a53c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:45.215672 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:45.219437 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d950fa9f-1235-47c7-a397-80e37d28a53c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:45.258555 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39820671-2649-4ec6-892e-3a18d56fdbda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:45.259386 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39820671-2649-4ec6-892e-3a18d56fdbda None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:45.259675 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39820671-2649-4ec6-892e-3a18d56fdbda None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:45.259950 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39820671-2649-4ec6-892e-3a18d56fdbda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:45.270675 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39820671-2649-4ec6-892e-3a18d56fdbda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:45.272141 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39820671-2649-4ec6-892e-3a18d56fdbda None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:45.289237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-39820671-2649-4ec6-892e-3a18d56fdbda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:45.289237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:45.289237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:45.289237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:45.289237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:45.289237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:45.289237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:45.289237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:45.289237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:45.289237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:45.289237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:45.289237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:45.289237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:03:45.289237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:03:45.289237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:03:45.290149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:03:45.290149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:45.290149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:45.290149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:45.290149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:45.290149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:45.290149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:45.290149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:45.290149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:45.290149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:45.290149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:45.290149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:45.290149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:45.290149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:45.290149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:45.290149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:45.290149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:45.291068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:03:45.291068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:03:45.291068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:45.291068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:45.291068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 525/1068] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:45.296119 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822be948-0343-4739-b620-7a98006f5c6d None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:45.399544 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:45.407877 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822be948-0343-4739-b620-7a98006f5c6d None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:45.443095 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a25b9e2-fa1c-42dc-b85a-57e459336e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:45.443789 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a25b9e2-fa1c-42dc-b85a-57e459336e70 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:45.444073 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a25b9e2-fa1c-42dc-b85a-57e459336e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:45.444350 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a25b9e2-fa1c-42dc-b85a-57e459336e70 None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:45.445073 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a25b9e2-fa1c-42dc-b85a-57e459336e70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:45.445381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a25b9e2-fa1c-42dc-b85a-57e459336e70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:45.446733 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a25b9e2-fa1c-42dc-b85a-57e459336e70 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:45.463521 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 544/1069] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:45.465019 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39820671-2649-4ec6-892e-3a18d56fdbda None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:45.569015 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39820671-2649-4ec6-892e-3a18d56fdbda None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:45.601823 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adfe62d-5a79-4778-bc69-eb3211c67aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:45.602605 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adfe62d-5a79-4778-bc69-eb3211c67aae None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:45.602891 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adfe62d-5a79-4778-bc69-eb3211c67aae None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:45.603170 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adfe62d-5a79-4778-bc69-eb3211c67aae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:45.610512 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adfe62d-5a79-4778-bc69-eb3211c67aae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:45.611980 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adfe62d-5a79-4778-bc69-eb3211c67aae None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:45.626400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-8adfe62d-5a79-4778-bc69-eb3211c67aae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:03:45.626400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:45.626400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:45.626400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:45.626400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:45.626400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:45.626400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:45.626400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:45.626400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:45.626400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:45.626400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:45.626400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:45.626400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:03:45.626400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:03:45.626400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:03:45.627384 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:03:45.627384 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:45.627384 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:45.627384 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:45.627384 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:45.627384 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:45.627384 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:45.627384 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:45.627384 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:45.627384 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:45.627384 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:45.627384 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:45.627384 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:45.627384 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:45.627384 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:45.627384 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:45.627384 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:45.633052 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:03:45.633052 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:03:45.633052 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:03:45.633052 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:03:45.633052 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:03:45.633052 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:03:45.633052 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:03:45.633052 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:45.633052 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 526/1070] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:03:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:45.635176 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a25b9e2-fa1c-42dc-b85a-57e459336e70 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:45.751973 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a25b9e2-fa1c-42dc-b85a-57e459336e70 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:45.785657 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe3cd95-c793-467f-9d84-72e65f79053d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:45.786348 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3cd95-c793-467f-9d84-72e65f79053d None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:45.786604 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3cd95-c793-467f-9d84-72e65f79053d None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:45.786880 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3cd95-c793-467f-9d84-72e65f79053d None admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:45.787699 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe3cd95-c793-467f-9d84-72e65f79053d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:45.788011 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe3cd95-c793-467f-9d84-72e65f79053d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:45.789431 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe3cd95-c793-467f-9d84-72e65f79053d None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:45.806853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 545/1071] 10.222.0.41 () {56 vars in 1148 bytes} [Fri Jun 3 16:03:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:45.808180 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adfe62d-5a79-4778-bc69-eb3211c67aae None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:45.823341 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:45.909117 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adfe62d-5a79-4778-bc69-eb3211c67aae None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:45.941982 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4bbec-1ca7-482a-8590-0a79caacef67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:45.942858 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4bbec-1ca7-482a-8590-0a79caacef67 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:45.943137 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4bbec-1ca7-482a-8590-0a79caacef67 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:45.943438 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4bbec-1ca7-482a-8590-0a79caacef67 None admin] PATH_INFO: `/v3/projects/c274254e510a468e9acbecc6298301cc/users/2fd08ea39ee94b92ac6763608e69f158/roles/3552bc334c464cd19e06a671ad16af6c` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:45.945599 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d4bbec-1ca7-482a-8590-0a79caacef67 None admin] RBAC: Authorizing `identity:create_grant(project_id=c274254e510a468e9acbecc6298301cc, user_id=2fd08ea39ee94b92ac6763608e69f158, role_id=3552bc334c464cd19e06a671ad16af6c)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:45.947022 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d4bbec-1ca7-482a-8590-0a79caacef67 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:45.967335 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 527/1072] 10.222.0.41 () {58 vars in 1439 bytes} [Fri Jun 3 16:03:45 2022] PUT /identity/v3/projects/c274254e510a468e9acbecc6298301cc/users/2fd08ea39ee94b92ac6763608e69f158/roles/3552bc334c464cd19e06a671ad16af6c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:03:46.612477 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb21354-9916-4c54-84d8-0705ac2c4364 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:46.613010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb21354-9916-4c54-84d8-0705ac2c4364 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:46.613010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb21354-9916-4c54-84d8-0705ac2c4364 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:46.615854 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 546/1073] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:46.621629 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ad718-8b0b-4f00-b5b6-b4023cc35948 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:46.622104 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ad718-8b0b-4f00-b5b6-b4023cc35948 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:46.622209 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ad718-8b0b-4f00-b5b6-b4023cc35948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:46.639892 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-8b2ad718-8b0b-4f00-b5b6-b4023cc35948 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:46.641653 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8b2ad718-8b0b-4f00-b5b6-b4023cc35948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:46.710185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 528/1074] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:03:46.732570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb21354-9916-4c54-84d8-0705ac2c4364 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:46.749425 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:46.839417 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb21354-9916-4c54-84d8-0705ac2c4364 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:46.876009 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926926a0-6cff-4f25-8162-ff4aa9badb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:46.876735 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926926a0-6cff-4f25-8162-ff4aa9badb00 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:46.877016 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926926a0-6cff-4f25-8162-ff4aa9badb00 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:46.877291 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926926a0-6cff-4f25-8162-ff4aa9badb00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:46.887547 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926926a0-6cff-4f25-8162-ff4aa9badb00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:03:46.895220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-926926a0-6cff-4f25-8162-ff4aa9badb00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:03:46.895220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:46.895220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:46.895220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:46.895220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:46.895220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:46.895220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:46.895220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:46.895220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:46.895220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:46.895220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:46.895220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:46.895220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:03:46.895220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:03:46.895220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:03:46.895220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:03:46.896269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:03:46.896269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:03:46.896269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:03:46.896269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:03:46.896269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:46.896269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:46.896269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:46.896269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:46.896269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:46.896269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:46.896269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:46.896269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:46.896269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:46.896269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:46.896269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:46.896269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:46.897129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:46.897129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:46.897129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:46.897129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:46.897129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:03:46.897129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:03:46.897129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:03:46.897129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:03:46.897129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:03:46.897129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:03:46.897129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:03:46.897129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:46.899656 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 547/1075] 10.222.0.41 () {56 vars in 1096 bytes} [Fri Jun 3 16:03:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:46.902821 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ad718-8b0b-4f00-b5b6-b4023cc35948 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:46.905186 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:47.008537 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ad718-8b0b-4f00-b5b6-b4023cc35948 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:47.043670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42c217c-9722-40ee-9dd9-6f735d9c5f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:47.044600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c217c-9722-40ee-9dd9-6f735d9c5f46 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:47.045061 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c217c-9722-40ee-9dd9-6f735d9c5f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:47.045612 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c217c-9722-40ee-9dd9-6f735d9c5f46 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:47.046524 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42c217c-9722-40ee-9dd9-6f735d9c5f46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:47.047004 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42c217c-9722-40ee-9dd9-6f735d9c5f46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:47.048577 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42c217c-9722-40ee-9dd9-6f735d9c5f46 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:47.061703 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 529/1076] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:03:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:47.062883 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926926a0-6cff-4f25-8162-ff4aa9badb00 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:47.141974 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:47.168051 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926926a0-6cff-4f25-8162-ff4aa9badb00 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:47.202185 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66568c84-2b05-462f-8d5f-43e5fb1581b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:47.203450 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66568c84-2b05-462f-8d5f-43e5fb1581b2 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:47.204473 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66568c84-2b05-462f-8d5f-43e5fb1581b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:47.204945 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66568c84-2b05-462f-8d5f-43e5fb1581b2 None admin] PATH_INFO: `/v3/users/ff5049ddc4244e50b65cd0e5bf221639` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:47.206366 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66568c84-2b05-462f-8d5f-43e5fb1581b2 None admin] RBAC: Authorizing `identity:get_user(user_id=ff5049ddc4244e50b65cd0e5bf221639)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:47.208015 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66568c84-2b05-462f-8d5f-43e5fb1581b2 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:47.214257 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 548/1077] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:03:47 2022] GET /identity/v3/users/ff5049ddc4244e50b65cd0e5bf221639 => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:47.215823 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42c217c-9722-40ee-9dd9-6f735d9c5f46 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:47.245599 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466319b-a113-4d09-9536-6691e2c4ef7c None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:47.245898 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466319b-a113-4d09-9536-6691e2c4ef7c None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:47.246184 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466319b-a113-4d09-9536-6691e2c4ef7c None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:47.247735 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 549/1078] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 16:03:47.253464 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b1e3f-7127-4268-8dfe-49b61ed33c08 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:47.253756 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b1e3f-7127-4268-8dfe-49b61ed33c08 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:47.254043 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b1e3f-7127-4268-8dfe-49b61ed33c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:47.271219 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-302b1e3f-7127-4268-8dfe-49b61ed33c08 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:47.272449 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-302b1e3f-7127-4268-8dfe-49b61ed33c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:47.285749 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:47.326854 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42c217c-9722-40ee-9dd9-6f735d9c5f46 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:47.341955 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 550/1079] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 16:03:47.363147 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398339d-8f2e-4d5f-bbc7-d8bf46dfcf8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:47.363938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b1e3f-7127-4268-8dfe-49b61ed33c08 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:47.364421 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398339d-8f2e-4d5f-bbc7-d8bf46dfcf8d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:47.364858 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398339d-8f2e-4d5f-bbc7-d8bf46dfcf8d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:47.365298 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398339d-8f2e-4d5f-bbc7-d8bf46dfcf8d None admin] PATH_INFO: `/v3/projects/24b01e54c67341d490411f88e85a39f1` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:47.367682 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398339d-8f2e-4d5f-bbc7-d8bf46dfcf8d None admin] RBAC: Authorizing `identity:get_project(project_id=24b01e54c67341d490411f88e85a39f1)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:47.369126 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398339d-8f2e-4d5f-bbc7-d8bf46dfcf8d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:47.370975 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 530/1080] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:03:47 2022] GET /identity/v3/projects/24b01e54c67341d490411f88e85a39f1 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:47.376974 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398339d-8f2e-4d5f-bbc7-d8bf46dfcf8d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:47.441361 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:47.471841 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b1e3f-7127-4268-8dfe-49b61ed33c08 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:47.483593 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398339d-8f2e-4d5f-bbc7-d8bf46dfcf8d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:47.510701 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81370b69-74c1-4abe-9d51-c5a0b0097bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:47.511383 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81370b69-74c1-4abe-9d51-c5a0b0097bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:47.511671 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81370b69-74c1-4abe-9d51-c5a0b0097bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:47.511954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81370b69-74c1-4abe-9d51-c5a0b0097bf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:47.514694 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:47.521181 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ccaa7d-2a27-4cc9-a994-55a2bf030d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:47.522055 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ccaa7d-2a27-4cc9-a994-55a2bf030d68 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:47.522341 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ccaa7d-2a27-4cc9-a994-55a2bf030d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:47.522616 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ccaa7d-2a27-4cc9-a994-55a2bf030d68 None admin] PATH_INFO: `/v3/projects/24b01e54c67341d490411f88e85a39f1/users/ff5049ddc4244e50b65cd0e5bf221639/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:47.524624 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ccaa7d-2a27-4cc9-a994-55a2bf030d68 None admin] RBAC: Authorizing `identity:create_grant(project_id=24b01e54c67341d490411f88e85a39f1, user_id=ff5049ddc4244e50b65cd0e5bf221639, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:47.526110 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ccaa7d-2a27-4cc9-a994-55a2bf030d68 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:47.540336 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 531/1081] 10.222.0.41 () {58 vars in 1439 bytes} [Fri Jun 3 16:03:47 2022] PUT /identity/v3/projects/24b01e54c67341d490411f88e85a39f1/users/ff5049ddc4244e50b65cd0e5bf221639/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:03:47.588665 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81370b69-74c1-4abe-9d51-c5a0b0097bf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:47.590158 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81370b69-74c1-4abe-9d51-c5a0b0097bf5 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:47.679737 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:47.722161 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 551/1082] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:03:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 03 16:03:47.723393 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ccaa7d-2a27-4cc9-a994-55a2bf030d68 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:47.832744 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ccaa7d-2a27-4cc9-a994-55a2bf030d68 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:47.866612 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee27e4-a3b0-48df-963e-5755b27167c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:47.867425 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee27e4-a3b0-48df-963e-5755b27167c6 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:47.867712 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee27e4-a3b0-48df-963e-5755b27167c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:47.867989 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee27e4-a3b0-48df-963e-5755b27167c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:47.879259 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee27e4-a3b0-48df-963e-5755b27167c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:47.880757 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee27e4-a3b0-48df-963e-5755b27167c6 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:47.899727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-dfee27e4-a3b0-48df-963e-5755b27167c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:47.899727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:47.899727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:47.899727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:47.899727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:47.899727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:47.899727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:47.899727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:47.899727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:47.899727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:47.899727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:47.899727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:47.899727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:03:47.899727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:03:47.899727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:03:47.900900 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:03:47.900900 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:47.900900 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:47.900900 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:47.900900 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:47.900900 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:47.900900 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:47.900900 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:47.900900 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:47.900900 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:47.900900 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:47.900900 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:47.900900 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:47.900900 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:47.900900 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:47.900900 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:47.900900 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:47.901995 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:03:47.901995 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:03:47.901995 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:47.901995 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:47.901995 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 532/1083] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:47.908850 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81370b69-74c1-4abe-9d51-c5a0b0097bf5 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:48.010511 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81370b69-74c1-4abe-9d51-c5a0b0097bf5 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:48.044540 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726fa56-a600-42e7-b0c0-9b20fb3def39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:48.045206 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726fa56-a600-42e7-b0c0-9b20fb3def39 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:48.045490 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726fa56-a600-42e7-b0c0-9b20fb3def39 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:48.045766 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726fa56-a600-42e7-b0c0-9b20fb3def39 None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:48.046486 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7726fa56-a600-42e7-b0c0-9b20fb3def39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:48.046801 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7726fa56-a600-42e7-b0c0-9b20fb3def39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:48.048235 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7726fa56-a600-42e7-b0c0-9b20fb3def39 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:48.060811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 552/1084] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:03:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:48.228260 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d38cc9-ceae-4385-9ed0-86b9726ad9fd None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:48.228545 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d38cc9-ceae-4385-9ed0-86b9726ad9fd None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:48.228826 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d38cc9-ceae-4385-9ed0-86b9726ad9fd None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:48.230208 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 533/1085] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:48.236287 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77afdb99-4e18-4538-84b4-34e96ba2066b None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:48.236589 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77afdb99-4e18-4538-84b4-34e96ba2066b None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:48.236867 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77afdb99-4e18-4538-84b4-34e96ba2066b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:48.255202 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-77afdb99-4e18-4538-84b4-34e96ba2066b None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:48.256428 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-77afdb99-4e18-4538-84b4-34e96ba2066b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:48.286575 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:48.329484 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 553/1086] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 16:03:48.353522 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d38cc9-ceae-4385-9ed0-86b9726ad9fd None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:48.405205 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:48.460399 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d38cc9-ceae-4385-9ed0-86b9726ad9fd None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:48.492987 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddffad7-fce4-42b2-be13-4c72ba65c11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:48.493785 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddffad7-fce4-42b2-be13-4c72ba65c11c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:48.494068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddffad7-fce4-42b2-be13-4c72ba65c11c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:48.494344 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddffad7-fce4-42b2-be13-4c72ba65c11c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:48.505111 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddffad7-fce4-42b2-be13-4c72ba65c11c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:03:48.513083 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-bddffad7-fce4-42b2-be13-4c72ba65c11c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 03 16:03:48.513083 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:48.513083 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:48.513083 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:48.513083 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:48.513083 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:48.513083 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:48.513083 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:48.513083 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:48.513083 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:48.513083 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:48.513083 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:48.513083 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:03:48.513083 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:03:48.513083 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:03:48.513083 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:03:48.514250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:03:48.514250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:03:48.514250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:03:48.514250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:03:48.514250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:48.514250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:48.514250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:48.514250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:48.514250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:48.514250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:48.514250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:48.514250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:48.514250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:48.514250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:48.514250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:48.514250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:48.520198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:48.520198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:48.520198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:48.520198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:48.520198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:03:48.520198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:03:48.520198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:03:48.520198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:03:48.520198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:03:48.520198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:03:48.520198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 03 16:03:48.520198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:48.520198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 534/1087] 10.222.0.41 () {56 vars in 1120 bytes} [Fri Jun 3 16:03:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:48.521112 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77afdb99-4e18-4538-84b4-34e96ba2066b None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:48.614977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77afdb99-4e18-4538-84b4-34e96ba2066b None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:48.646703 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193d65b-41c0-4ef6-89ed-14f057564696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:48.647384 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193d65b-41c0-4ef6-89ed-14f057564696 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:48.647669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193d65b-41c0-4ef6-89ed-14f057564696 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:48.647944 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193d65b-41c0-4ef6-89ed-14f057564696 None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:48.648677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f193d65b-41c0-4ef6-89ed-14f057564696 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:48.648981 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f193d65b-41c0-4ef6-89ed-14f057564696 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:48.650322 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f193d65b-41c0-4ef6-89ed-14f057564696 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:48.658664 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 554/1088] 10.222.0.41 () {56 vars in 1115 bytes} [Fri Jun 3 16:03:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:48.664179 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddffad7-fce4-42b2-be13-4c72ba65c11c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:48.763052 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddffad7-fce4-42b2-be13-4c72ba65c11c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:48.765363 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:48.797788 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1436ce-c74a-422d-9268-6d9946deeacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:48.798579 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1436ce-c74a-422d-9268-6d9946deeacb None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:48.798861 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1436ce-c74a-422d-9268-6d9946deeacb None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:48.799132 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1436ce-c74a-422d-9268-6d9946deeacb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:48.808941 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1436ce-c74a-422d-9268-6d9946deeacb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:48.810477 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1436ce-c74a-422d-9268-6d9946deeacb None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:48.827244 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-3d1436ce-c74a-422d-9268-6d9946deeacb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:48.827244 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:48.827244 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:48.827244 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:48.827244 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:48.827244 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:48.827244 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:48.827244 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:48.827244 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:48.827244 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:48.827244 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:48.827244 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:48.827244 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:03:48.827244 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:03:48.827244 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:03:48.828444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:03:48.828444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:48.828444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:48.828444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:48.828444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:48.828444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:48.828444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:48.828444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:48.828444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:48.828444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:48.828444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:48.828444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:48.828444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:48.828444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:48.828444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:48.828444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:48.828444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:48.829361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:03:48.829361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:03:48.829361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:48.829361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:48.829361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 535/1089] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:03:48.834070 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193d65b-41c0-4ef6-89ed-14f057564696 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:48.859272 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:48.932392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193d65b-41c0-4ef6-89ed-14f057564696 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:48.965441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885a446-2023-42e3-8a14-1c672f7daf22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:48.966096 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885a446-2023-42e3-8a14-1c672f7daf22 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:48.966410 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885a446-2023-42e3-8a14-1c672f7daf22 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:48.966689 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885a446-2023-42e3-8a14-1c672f7daf22 None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:48.967443 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885a446-2023-42e3-8a14-1c672f7daf22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:48.967756 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885a446-2023-42e3-8a14-1c672f7daf22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:48.969716 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885a446-2023-42e3-8a14-1c672f7daf22 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:48.981125 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 555/1090] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:03:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:48.986590 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1436ce-c74a-422d-9268-6d9946deeacb None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:49.082705 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1436ce-c74a-422d-9268-6d9946deeacb None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:49.114820 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddabe483-6696-44c6-84f5-184d0597674c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:49.115629 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddabe483-6696-44c6-84f5-184d0597674c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:49.115915 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddabe483-6696-44c6-84f5-184d0597674c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:49.116188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddabe483-6696-44c6-84f5-184d0597674c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:49.123407 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddabe483-6696-44c6-84f5-184d0597674c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:49.124829 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddabe483-6696-44c6-84f5-184d0597674c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:49.138294 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-ddabe483-6696-44c6-84f5-184d0597674c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 03 16:03:49.138294 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:49.138294 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:49.138294 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:49.138294 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:49.138294 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:49.138294 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:49.138294 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:49.138294 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:49.138294 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:49.138294 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:49.138294 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:49.138294 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:03:49.138294 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:03:49.138294 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:03:49.138294 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:03:49.139248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:49.139248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:49.139248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:03:49.139248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:03:49.139248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:03:49.139248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:03:49.139248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:49.139248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:49.139248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:49.139248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:49.139248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:49.139248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:49.139248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:49.139248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:49.139248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:49.139248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:49.139248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:49.139248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:49.140243 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:49.140243 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:49.140243 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:03:49.140243 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:03:49.140243 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:03:49.140243 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:03:49.140243 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:03:49.140243 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:03:49.140243 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 03 16:03:49.140243 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:49.140243 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 536/1091] 10.222.0.41 () {56 vars in 1093 bytes} [Fri Jun 3 16:03:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:03:49.145666 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885a446-2023-42e3-8a14-1c672f7daf22 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:49.229849 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:49.246144 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885a446-2023-42e3-8a14-1c672f7daf22 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:49.280009 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51b1b23-c3ea-4c18-ba51-da02024f196c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:49.280673 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b1b23-c3ea-4c18-ba51-da02024f196c None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:49.280961 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b1b23-c3ea-4c18-ba51-da02024f196c None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:49.281249 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b1b23-c3ea-4c18-ba51-da02024f196c None admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:49.282727 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51b1b23-c3ea-4c18-ba51-da02024f196c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:49.283038 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51b1b23-c3ea-4c18-ba51-da02024f196c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:49.284485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51b1b23-c3ea-4c18-ba51-da02024f196c None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:49.302115 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 556/1092] 10.222.0.41 () {56 vars in 1133 bytes} [Fri Jun 3 16:03:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:49.307778 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddabe483-6696-44c6-84f5-184d0597674c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:49.326326 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:49.406307 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddabe483-6696-44c6-84f5-184d0597674c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:49.438189 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a646988-d83c-43b6-bdc8-6168e81674f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:49.438832 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a646988-d83c-43b6-bdc8-6168e81674f4 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:49.439115 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a646988-d83c-43b6-bdc8-6168e81674f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:49.439426 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a646988-d83c-43b6-bdc8-6168e81674f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:49.507984 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a646988-d83c-43b6-bdc8-6168e81674f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:49.509351 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a646988-d83c-43b6-bdc8-6168e81674f4 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:49.526347 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:49.607151 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 537/1093] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:03:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 03 16:03:49.612823 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51b1b23-c3ea-4c18-ba51-da02024f196c None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:49.717244 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51b1b23-c3ea-4c18-ba51-da02024f196c None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:49.749623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325e596-1b4b-4226-b4a4-c089cc42f324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:49.751016 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325e596-1b4b-4226-b4a4-c089cc42f324 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:49.751315 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325e596-1b4b-4226-b4a4-c089cc42f324 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:49.751601 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325e596-1b4b-4226-b4a4-c089cc42f324 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:49.761371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a325e596-1b4b-4226-b4a4-c089cc42f324 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:49.762730 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a325e596-1b4b-4226-b4a4-c089cc42f324 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:49.764423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:49.780724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-a325e596-1b4b-4226-b4a4-c089cc42f324 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:49.780724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:49.780724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:49.780724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:49.780724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:49.780724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:49.780724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:49.780724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:49.780724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:49.780724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:49.780724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:49.780724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:49.780724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:03:49.780724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:03:49.780724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:03:49.781790 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:03:49.781790 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:49.781790 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:49.781790 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:49.781790 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:49.781790 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:49.781790 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:49.781790 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:49.781790 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:49.781790 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:49.781790 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:49.781790 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:49.781790 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:49.781790 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:49.781790 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:49.781790 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:49.781790 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:49.782692 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:03:49.782692 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:03:49.782692 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:03:49.782692 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:49.782692 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 557/1094] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:03:49.787506 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a646988-d83c-43b6-bdc8-6168e81674f4 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:49.801972 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f409bb9-0acf-4f20-9c7d-63c96ca1ff1f None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:49.802269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f409bb9-0acf-4f20-9c7d-63c96ca1ff1f None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:49.802553 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f409bb9-0acf-4f20-9c7d-63c96ca1ff1f None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:49.803943 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 558/1095] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:49.808921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7bc21f-2c19-4985-b8a8-edc1173f4c97 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:49.809208 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7bc21f-2c19-4985-b8a8-edc1173f4c97 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:49.809491 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7bc21f-2c19-4985-b8a8-edc1173f4c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:49.826047 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-ff7bc21f-2c19-4985-b8a8-edc1173f4c97 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:49.827234 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-ff7bc21f-2c19-4985-b8a8-edc1173f4c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:49.886169 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a646988-d83c-43b6-bdc8-6168e81674f4 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:49.892757 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 559/1096] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 16:03:49.902356 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:49.916930 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7bc21f-2c19-4985-b8a8-edc1173f4c97 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:49.922147 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4202d539-9f20-4eae-bd00-26f36b8b1993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:49.922812 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202d539-9f20-4eae-bd00-26f36b8b1993 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:49.923103 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202d539-9f20-4eae-bd00-26f36b8b1993 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:49.923414 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202d539-9f20-4eae-bd00-26f36b8b1993 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:49.924186 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4202d539-9f20-4eae-bd00-26f36b8b1993 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:49.924499 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4202d539-9f20-4eae-bd00-26f36b8b1993 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:49.925872 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4202d539-9f20-4eae-bd00-26f36b8b1993 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:49.937201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 538/1097] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:03:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:49.942355 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4202d539-9f20-4eae-bd00-26f36b8b1993 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:50.014478 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7bc21f-2c19-4985-b8a8-edc1173f4c97 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:50.038248 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:50.038390 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4202d539-9f20-4eae-bd00-26f36b8b1993 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:50.049920 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b6611-14cb-4cd1-b558-4260d08b50a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:50.050711 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b6611-14cb-4cd1-b558-4260d08b50a7 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:50.050998 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b6611-14cb-4cd1-b558-4260d08b50a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:50.051273 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b6611-14cb-4cd1-b558-4260d08b50a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:50.061211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9b6611-14cb-4cd1-b558-4260d08b50a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:03:50.068825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-8a9b6611-14cb-4cd1-b558-4260d08b50a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:03:50.068825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:50.068825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:50.068825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:50.068825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:50.068825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:50.068825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:50.068825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:50.068825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:50.068825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:50.068825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:50.068825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:50.068825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:03:50.068825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:03:50.068825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:03:50.068825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:03:50.069902 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:03:50.069902 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:03:50.069902 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:03:50.069902 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:03:50.069902 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:50.069902 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:50.069902 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:50.069902 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:50.069902 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:50.069902 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:50.069902 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:50.069902 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:50.069902 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:50.069902 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:50.069902 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:50.069902 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:50.070752 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:50.070752 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:50.070752 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:50.070752 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:50.070752 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:03:50.070752 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:03:50.070752 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:03:50.070752 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:03:50.070752 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:03:50.070752 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:03:50.070752 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:03:50.070752 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:03:50.070752 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 560/1098] 10.222.0.41 () {56 vars in 1096 bytes} [Fri Jun 3 16:03:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:03:50.075795 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47761cf6-b655-48b4-b135-dead26ea60cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:50.075944 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b6611-14cb-4cd1-b558-4260d08b50a7 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:50.076576 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47761cf6-b655-48b4-b135-dead26ea60cb None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:50.076855 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47761cf6-b655-48b4-b135-dead26ea60cb None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:50.077123 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47761cf6-b655-48b4-b135-dead26ea60cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:50.085517 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47761cf6-b655-48b4-b135-dead26ea60cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:50.087334 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47761cf6-b655-48b4-b135-dead26ea60cb None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:50.101880 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-47761cf6-b655-48b4-b135-dead26ea60cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:03:50.101880 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:03:50.101880 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:03:50.101880 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:03:50.101880 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:03:50.101880 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:03:50.101880 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:03:50.101880 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:03:50.101880 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:03:50.101880 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:03:50.101880 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:03:50.101880 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:03:50.101880 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:03:50.101880 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:03:50.101880 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:03:50.102899 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:03:50.102899 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:03:50.102899 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:03:50.102899 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:03:50.102899 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:03:50.102899 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:03:50.102899 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:03:50.102899 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:03:50.102899 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:03:50.102899 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:03:50.102899 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:03:50.102899 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:03:50.102899 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:03:50.102899 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:03:50.102899 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:03:50.102899 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:03:50.102899 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:03:50.103832 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:03:50.103832 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:03:50.103832 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:03:50.103832 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:03:50.103832 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:03:50.103832 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:03:50.103832 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:03:50.103832 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:03:50.107549 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 539/1099] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:03:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:03:50.108715 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47761cf6-b655-48b4-b135-dead26ea60cb None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:50.144031 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:50.174340 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b6611-14cb-4cd1-b558-4260d08b50a7 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:50.207303 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ba983-7e7f-4655-9876-caa2a05db2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:50.207984 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ba983-7e7f-4655-9876-caa2a05db2ca None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:50.208110 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47761cf6-b655-48b4-b135-dead26ea60cb None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:50.208270 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ba983-7e7f-4655-9876-caa2a05db2ca None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:50.208545 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ba983-7e7f-4655-9876-caa2a05db2ca None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:50.209275 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ba983-7e7f-4655-9876-caa2a05db2ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:50.209582 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ba983-7e7f-4655-9876-caa2a05db2ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:50.210932 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ba983-7e7f-4655-9876-caa2a05db2ca None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:50.219200 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 561/1100] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:03:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:50.224846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ba983-7e7f-4655-9876-caa2a05db2ca None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:50.245269 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40875ba6-4d65-4d11-a4b6-e3561a91cfd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:50.245922 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40875ba6-4d65-4d11-a4b6-e3561a91cfd0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:50.246236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40875ba6-4d65-4d11-a4b6-e3561a91cfd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:50.246517 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40875ba6-4d65-4d11-a4b6-e3561a91cfd0 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:50.247315 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40875ba6-4d65-4d11-a4b6-e3561a91cfd0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:50.247646 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40875ba6-4d65-4d11-a4b6-e3561a91cfd0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:50.249088 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40875ba6-4d65-4d11-a4b6-e3561a91cfd0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:50.261130 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 540/1101] 10.222.0.41 () {56 vars in 1148 bytes} [Fri Jun 3 16:03:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:50.270506 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40875ba6-4d65-4d11-a4b6-e3561a91cfd0 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:50.272965 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:50.325920 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ba983-7e7f-4655-9876-caa2a05db2ca None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:50.359807 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4705aed7-3e0e-4e03-bc7b-744df9e21cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:50.360670 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705aed7-3e0e-4e03-bc7b-744df9e21cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:50.360949 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705aed7-3e0e-4e03-bc7b-744df9e21cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:50.361223 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705aed7-3e0e-4e03-bc7b-744df9e21cb7 None admin] PATH_INFO: `/v3/users/ff5049ddc4244e50b65cd0e5bf221639` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:50.361666 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:50.362524 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4705aed7-3e0e-4e03-bc7b-744df9e21cb7 None admin] RBAC: Authorizing `identity:get_user(user_id=ff5049ddc4244e50b65cd0e5bf221639)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:50.364132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4705aed7-3e0e-4e03-bc7b-744df9e21cb7 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:50.365732 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 562/1102] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:03:50 2022] GET /identity/v3/users/ff5049ddc4244e50b65cd0e5bf221639 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:50.367993 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40875ba6-4d65-4d11-a4b6-e3561a91cfd0 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:50.371257 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4705aed7-3e0e-4e03-bc7b-744df9e21cb7 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:50.401645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa003495-1f53-41bc-a375-bc56fb07df77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:50.402362 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa003495-1f53-41bc-a375-bc56fb07df77 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:50.402972 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa003495-1f53-41bc-a375-bc56fb07df77 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:50.403342 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa003495-1f53-41bc-a375-bc56fb07df77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:50.404168 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa003495-1f53-41bc-a375-bc56fb07df77 None admin] RBAC: Adding query filter params (role.id=3552bc334c464cd19e06a671ad16af6c, scope.project.id=c274254e510a468e9acbecc6298301cc, user.id=2fd08ea39ee94b92ac6763608e69f158) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:50.404481 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa003495-1f53-41bc-a375-bc56fb07df77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:50.405904 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa003495-1f53-41bc-a375-bc56fb07df77 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:50.411716 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 541/1103] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:03:50 2022] GET /identity/v3/role_assignments?user.id=2fd08ea39ee94b92ac6763608e69f158&scope.project.id=c274254e510a468e9acbecc6298301cc&role.id=3552bc334c464cd19e06a671ad16af6c => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:50.474392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4705aed7-3e0e-4e03-bc7b-744df9e21cb7 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:50.490531 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:50.512813 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186473c8-30c6-42f7-8045-64400cd97ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:50.513481 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186473c8-30c6-42f7-8045-64400cd97ee0 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:50.513767 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186473c8-30c6-42f7-8045-64400cd97ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:50.514046 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186473c8-30c6-42f7-8045-64400cd97ee0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:50.584631 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186473c8-30c6-42f7-8045-64400cd97ee0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:50.586006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186473c8-30c6-42f7-8045-64400cd97ee0 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:50.683976 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 563/1104] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:03:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 03 16:03:50.690496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa003495-1f53-41bc-a375-bc56fb07df77 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:50.792385 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa003495-1f53-41bc-a375-bc56fb07df77 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:50.826288 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15332229-204a-4579-bb5f-0aa2ca06d070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:50.827148 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15332229-204a-4579-bb5f-0aa2ca06d070 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:50.827465 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15332229-204a-4579-bb5f-0aa2ca06d070 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:50.827742 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15332229-204a-4579-bb5f-0aa2ca06d070 None admin] PATH_INFO: `/v3/projects/24b01e54c67341d490411f88e85a39f1` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:50.828937 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15332229-204a-4579-bb5f-0aa2ca06d070 None admin] RBAC: Authorizing `identity:get_project(project_id=24b01e54c67341d490411f88e85a39f1)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:50.830374 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15332229-204a-4579-bb5f-0aa2ca06d070 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:50.832093 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 542/1105] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:03:50 2022] GET /identity/v3/projects/24b01e54c67341d490411f88e85a39f1 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:50.838505 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186473c8-30c6-42f7-8045-64400cd97ee0 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:50.856060 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:50.946147 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186473c8-30c6-42f7-8045-64400cd97ee0 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:50.980399 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5425de2-fbdc-4a17-b21e-ae64c7cd8b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:50.981049 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5425de2-fbdc-4a17-b21e-ae64c7cd8b64 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:50.981331 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5425de2-fbdc-4a17-b21e-ae64c7cd8b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:50.981605 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5425de2-fbdc-4a17-b21e-ae64c7cd8b64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:50.982466 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5425de2-fbdc-4a17-b21e-ae64c7cd8b64 None admin] RBAC: Adding query filter params (role.id=75936a608b8149099ff54e317f3a6740, scope.project.id=24b01e54c67341d490411f88e85a39f1, user.id=ff5049ddc4244e50b65cd0e5bf221639) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:50.982778 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5425de2-fbdc-4a17-b21e-ae64c7cd8b64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:50.984225 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5425de2-fbdc-4a17-b21e-ae64c7cd8b64 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:50.991063 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 564/1106] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:03:50 2022] GET /identity/v3/role_assignments?user.id=ff5049ddc4244e50b65cd0e5bf221639&scope.project.id=24b01e54c67341d490411f88e85a39f1&role.id=75936a608b8149099ff54e317f3a6740 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:53.232596 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1700221-3f90-4972-9c7d-0a8f0dd9fb83 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:53.233128 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1700221-3f90-4972-9c7d-0a8f0dd9fb83 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:53.233128 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1700221-3f90-4972-9c7d-0a8f0dd9fb83 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:53.234532 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 543/1107] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:53.240999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3133e2-c1b7-44d6-a651-345cd91711d9 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:53.241410 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3133e2-c1b7-44d6-a651-345cd91711d9 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:53.241558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3133e2-c1b7-44d6-a651-345cd91711d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:53.258540 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-fc3133e2-c1b7-44d6-a651-345cd91711d9 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:53.259741 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-fc3133e2-c1b7-44d6-a651-345cd91711d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:53.322797 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 565/1108] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 16:03:53.348064 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1700221-3f90-4972-9c7d-0a8f0dd9fb83 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:53.376412 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:53.447204 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1700221-3f90-4972-9c7d-0a8f0dd9fb83 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:53.479231 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f083bb-eeee-417e-bb68-102e3c4abe35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:53.480036 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f083bb-eeee-417e-bb68-102e3c4abe35 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:53.480325 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f083bb-eeee-417e-bb68-102e3c4abe35 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:53.480696 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f083bb-eeee-417e-bb68-102e3c4abe35 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:53.481872 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f083bb-eeee-417e-bb68-102e3c4abe35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:53.483372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f083bb-eeee-417e-bb68-102e3c4abe35 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:53.485443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 544/1109] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:03:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:53.491064 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3133e2-c1b7-44d6-a651-345cd91711d9 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:53.501528 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:53.586940 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3133e2-c1b7-44d6-a651-345cd91711d9 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:53.619090 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd502d6-1c00-4f8c-8f24-9f9d1a4a087f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:53.619774 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd502d6-1c00-4f8c-8f24-9f9d1a4a087f None admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:53.620056 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd502d6-1c00-4f8c-8f24-9f9d1a4a087f None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:53.620328 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd502d6-1c00-4f8c-8f24-9f9d1a4a087f None admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:53.621031 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd502d6-1c00-4f8c-8f24-9f9d1a4a087f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:53.622434 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd502d6-1c00-4f8c-8f24-9f9d1a4a087f None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:53.642001 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 566/1110] 10.222.0.41 () {60 vars in 1132 bytes} [Fri Jun 3 16:03:53 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:03:55.898951 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460fe26-e744-4121-b78b-47fdc990311a None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:55.899513 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460fe26-e744-4121-b78b-47fdc990311a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:55.899603 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460fe26-e744-4121-b78b-47fdc990311a None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:55.900912 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 545/1111] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:55.907144 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfe974-03b3-4bf1-bbf8-9ac5389d2b19 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:55.907609 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfe974-03b3-4bf1-bbf8-9ac5389d2b19 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:55.907709 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfe974-03b3-4bf1-bbf8-9ac5389d2b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:55.924516 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-eedfe974-03b3-4bf1-bbf8-9ac5389d2b19 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:55.925678 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-eedfe974-03b3-4bf1-bbf8-9ac5389d2b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:55.982079 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:55.991913 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 567/1112] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 16:03:56.016075 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6460fe26-e744-4121-b78b-47fdc990311a None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:56.108463 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:56.115379 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6460fe26-e744-4121-b78b-47fdc990311a None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:56.148038 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01748f93-a5ed-4e7b-b5f8-912442e010c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:56.148880 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01748f93-a5ed-4e7b-b5f8-912442e010c8 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:56.149212 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01748f93-a5ed-4e7b-b5f8-912442e010c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:56.149439 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01748f93-a5ed-4e7b-b5f8-912442e010c8 None admin] PATH_INFO: `/v3/roles/2248ac4d70c34e459b8942dec0d7656b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:56.150893 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01748f93-a5ed-4e7b-b5f8-912442e010c8 None admin] RBAC: Authorizing `identity:get_role(role_id=2248ac4d70c34e459b8942dec0d7656b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:56.152282 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01748f93-a5ed-4e7b-b5f8-912442e010c8 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:56.153885 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 546/1113] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:03:56 2022] GET /identity/v3/roles/2248ac4d70c34e459b8942dec0d7656b => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:56.159996 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedfe974-03b3-4bf1-bbf8-9ac5389d2b19 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:56.255952 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedfe974-03b3-4bf1-bbf8-9ac5389d2b19 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:56.288139 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807f4551-8e5f-417c-8d9e-9cd0170c4254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:56.288996 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f4551-8e5f-417c-8d9e-9cd0170c4254 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:56.289277 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f4551-8e5f-417c-8d9e-9cd0170c4254 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:56.289551 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f4551-8e5f-417c-8d9e-9cd0170c4254 None admin] PATH_INFO: `/v3/users/2976362dbeef430aa4a47aac4ce7e381` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:56.302454 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807f4551-8e5f-417c-8d9e-9cd0170c4254 None admin] RBAC: Authorizing `identity:get_user(user_id=2976362dbeef430aa4a47aac4ce7e381)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:56.303922 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807f4551-8e5f-417c-8d9e-9cd0170c4254 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:56.305530 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 568/1114] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:03:56 2022] GET /identity/v3/users/2976362dbeef430aa4a47aac4ce7e381 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:03:56.311360 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01748f93-a5ed-4e7b-b5f8-912442e010c8 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:56.407265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01748f93-a5ed-4e7b-b5f8-912442e010c8 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:56.439026 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaa8f1f-f15e-437b-9b20-5bc42a06cce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:56.439673 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa8f1f-f15e-437b-9b20-5bc42a06cce9 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:56.439955 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa8f1f-f15e-437b-9b20-5bc42a06cce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:56.440228 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa8f1f-f15e-437b-9b20-5bc42a06cce9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:56.469573 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:56.504171 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaa8f1f-f15e-437b-9b20-5bc42a06cce9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:56.505591 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaa8f1f-f15e-437b-9b20-5bc42a06cce9 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:56.599649 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 547/1115] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:03:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 16:03:56.605416 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807f4551-8e5f-417c-8d9e-9cd0170c4254 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:56.655124 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:56.699281 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807f4551-8e5f-417c-8d9e-9cd0170c4254 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:56.732015 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c603da0-0909-4292-ae5e-61bcbc59d93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:56.732848 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c603da0-0909-4292-ae5e-61bcbc59d93a None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:56.733129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c603da0-0909-4292-ae5e-61bcbc59d93a None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:56.733398 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c603da0-0909-4292-ae5e-61bcbc59d93a None admin] PATH_INFO: `/v3/projects/24b01e54c67341d490411f88e85a39f1` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:56.734581 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c603da0-0909-4292-ae5e-61bcbc59d93a None admin] RBAC: Authorizing `identity:get_project(project_id=24b01e54c67341d490411f88e85a39f1)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:56.736011 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c603da0-0909-4292-ae5e-61bcbc59d93a None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:56.737640 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 569/1116] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:03:56 2022] GET /identity/v3/projects/24b01e54c67341d490411f88e85a39f1 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:56.743471 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaa8f1f-f15e-437b-9b20-5bc42a06cce9 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:56.802017 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:56.837519 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaa8f1f-f15e-437b-9b20-5bc42a06cce9 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:56.869299 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec66def-5585-471e-b95e-66140ecf1eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:56.869938 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec66def-5585-471e-b95e-66140ecf1eec None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:56.870228 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec66def-5585-471e-b95e-66140ecf1eec None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:56.870508 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec66def-5585-471e-b95e-66140ecf1eec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:56.871361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec66def-5585-471e-b95e-66140ecf1eec None admin] RBAC: Adding query filter params (role.id=2248ac4d70c34e459b8942dec0d7656b, scope.project.id=24b01e54c67341d490411f88e85a39f1, user.id=2976362dbeef430aa4a47aac4ce7e381) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:03:56.871680 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec66def-5585-471e-b95e-66140ecf1eec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:56.873114 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec66def-5585-471e-b95e-66140ecf1eec None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:56.878883 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 548/1117] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:03:56 2022] GET /identity/v3/role_assignments?user.id=2976362dbeef430aa4a47aac4ce7e381&scope.project.id=24b01e54c67341d490411f88e85a39f1&role.id=2248ac4d70c34e459b8942dec0d7656b => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:59.074517 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b96d5-f94e-499d-ae17-16870caa8c18 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:59.075046 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b96d5-f94e-499d-ae17-16870caa8c18 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:59.075153 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b96d5-f94e-499d-ae17-16870caa8c18 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:59.076483 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 570/1118] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:03:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:03:59.082178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617a9c4-84a8-4536-9bf3-e1bfc5ef35b1 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:59.082573 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617a9c4-84a8-4536-9bf3-e1bfc5ef35b1 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:59.082734 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617a9c4-84a8-4536-9bf3-e1bfc5ef35b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:59.099528 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-d617a9c4-84a8-4536-9bf3-e1bfc5ef35b1 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:03:59.100690 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d617a9c4-84a8-4536-9bf3-e1bfc5ef35b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:03:59.164753 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 549/1119] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:03:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:03:59.188392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b96d5-f94e-499d-ae17-16870caa8c18 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:59.288108 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b96d5-f94e-499d-ae17-16870caa8c18 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:59.306960 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:59.321175 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ba513-e6f5-450b-99c6-67681fcb1c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:59.322024 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ba513-e6f5-450b-99c6-67681fcb1c7d None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:59.322308 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ba513-e6f5-450b-99c6-67681fcb1c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:59.322583 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ba513-e6f5-450b-99c6-67681fcb1c7d None admin] PATH_INFO: `/v3/roles/2248ac4d70c34e459b8942dec0d7656b` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:59.324037 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ba513-e6f5-450b-99c6-67681fcb1c7d None admin] RBAC: Authorizing `identity:get_role(role_id=2248ac4d70c34e459b8942dec0d7656b)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:59.325442 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ba513-e6f5-450b-99c6-67681fcb1c7d None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:59.326742 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 571/1120] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:03:59 2022] GET /identity/v3/roles/2248ac4d70c34e459b8942dec0d7656b => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:59.333117 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617a9c4-84a8-4536-9bf3-e1bfc5ef35b1 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:59.388062 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:59.428002 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617a9c4-84a8-4536-9bf3-e1bfc5ef35b1 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:59.460574 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40174359-002e-4d0b-b4bb-e9658e968939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:59.461423 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40174359-002e-4d0b-b4bb-e9658e968939 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:59.461706 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40174359-002e-4d0b-b4bb-e9658e968939 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:59.461981 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40174359-002e-4d0b-b4bb-e9658e968939 None admin] PATH_INFO: `/v3/users/2976362dbeef430aa4a47aac4ce7e381` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:59.463143 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40174359-002e-4d0b-b4bb-e9658e968939 None admin] RBAC: Authorizing `identity:get_user(user_id=2976362dbeef430aa4a47aac4ce7e381)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:59.464616 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40174359-002e-4d0b-b4bb-e9658e968939 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:59.466220 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 550/1121] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:03:59 2022] GET /identity/v3/users/2976362dbeef430aa4a47aac4ce7e381 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:59.471716 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ba513-e6f5-450b-99c6-67681fcb1c7d None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:59.567813 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ba513-e6f5-450b-99c6-67681fcb1c7d None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:59.599569 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3033cc-c859-4934-93f0-91bb39ecc4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:59.600352 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3033cc-c859-4934-93f0-91bb39ecc4e8 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:59.600635 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3033cc-c859-4934-93f0-91bb39ecc4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:59.600910 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3033cc-c859-4934-93f0-91bb39ecc4e8 None admin] PATH_INFO: `/v3/projects/24b01e54c67341d490411f88e85a39f1` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:59.602050 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3033cc-c859-4934-93f0-91bb39ecc4e8 None admin] RBAC: Authorizing `identity:get_project(project_id=24b01e54c67341d490411f88e85a39f1)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:59.603487 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3033cc-c859-4934-93f0-91bb39ecc4e8 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:59.605173 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 572/1122] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:03:59 2022] GET /identity/v3/projects/24b01e54c67341d490411f88e85a39f1 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:03:59.611572 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40174359-002e-4d0b-b4bb-e9658e968939 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:03:59.706813 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40174359-002e-4d0b-b4bb-e9658e968939 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:03:59.730809 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:03:59.739838 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5bb6d6-ea9e-43fe-acb0-3a81a10c0bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:03:59.740712 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5bb6d6-ea9e-43fe-acb0-3a81a10c0bd0 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:03:59.740996 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5bb6d6-ea9e-43fe-acb0-3a81a10c0bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:03:59.741270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5bb6d6-ea9e-43fe-acb0-3a81a10c0bd0 None admin] PATH_INFO: `/v3/projects/24b01e54c67341d490411f88e85a39f1/users/2976362dbeef430aa4a47aac4ce7e381/roles/2248ac4d70c34e459b8942dec0d7656b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:03:59.743490 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5bb6d6-ea9e-43fe-acb0-3a81a10c0bd0 None admin] RBAC: Authorizing `identity:create_grant(project_id=24b01e54c67341d490411f88e85a39f1, user_id=2976362dbeef430aa4a47aac4ce7e381, role_id=2248ac4d70c34e459b8942dec0d7656b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:03:59.744901 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5bb6d6-ea9e-43fe-acb0-3a81a10c0bd0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:03:59.756628 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 551/1123] 10.222.0.41 () {58 vars in 1439 bytes} [Fri Jun 3 16:03:59 2022] PUT /identity/v3/projects/24b01e54c67341d490411f88e85a39f1/users/2976362dbeef430aa4a47aac4ce7e381/roles/2248ac4d70c34e459b8942dec0d7656b => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:04:01.948291 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f2394f-ee7c-436c-a730-e1ec0d30cf13 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:01.948815 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f2394f-ee7c-436c-a730-e1ec0d30cf13 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:01.948815 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f2394f-ee7c-436c-a730-e1ec0d30cf13 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:01.950195 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 573/1124] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:04:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:04:01.956072 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5524f88-6d95-4fa0-b9a3-0a52e6421931 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:01.956450 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5524f88-6d95-4fa0-b9a3-0a52e6421931 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:01.956718 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5524f88-6d95-4fa0-b9a3-0a52e6421931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:01.974436 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-d5524f88-6d95-4fa0-b9a3-0a52e6421931 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:04:01.975629 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d5524f88-6d95-4fa0-b9a3-0a52e6421931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:04:02.041312 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 552/1125] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:04:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:04:02.064294 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f2394f-ee7c-436c-a730-e1ec0d30cf13 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:02.119602 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:02.163597 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f2394f-ee7c-436c-a730-e1ec0d30cf13 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:02.196610 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36838974-7319-4b8e-a323-1eb0c5a898a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:02.197451 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36838974-7319-4b8e-a323-1eb0c5a898a4 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:02.197733 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36838974-7319-4b8e-a323-1eb0c5a898a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:02.197999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36838974-7319-4b8e-a323-1eb0c5a898a4 None admin] PATH_INFO: `/v3/roles/2248ac4d70c34e459b8942dec0d7656b` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:02.199527 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36838974-7319-4b8e-a323-1eb0c5a898a4 None admin] RBAC: Authorizing `identity:get_role(role_id=2248ac4d70c34e459b8942dec0d7656b)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:02.200828 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36838974-7319-4b8e-a323-1eb0c5a898a4 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:02.202159 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 574/1126] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:04:02 2022] GET /identity/v3/roles/2248ac4d70c34e459b8942dec0d7656b => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:04:02.208019 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5524f88-6d95-4fa0-b9a3-0a52e6421931 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:02.304654 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5524f88-6d95-4fa0-b9a3-0a52e6421931 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:02.320530 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:02.338634 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c245e2-bb41-4746-a17a-b2348cdc9037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:02.339515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c245e2-bb41-4746-a17a-b2348cdc9037 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:02.339798 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c245e2-bb41-4746-a17a-b2348cdc9037 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:02.340071 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c245e2-bb41-4746-a17a-b2348cdc9037 None admin] PATH_INFO: `/v3/users/2976362dbeef430aa4a47aac4ce7e381` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:02.341256 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c245e2-bb41-4746-a17a-b2348cdc9037 None admin] RBAC: Authorizing `identity:get_user(user_id=2976362dbeef430aa4a47aac4ce7e381)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:02.342679 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c245e2-bb41-4746-a17a-b2348cdc9037 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:02.344359 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 553/1127] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:04:02 2022] GET /identity/v3/users/2976362dbeef430aa4a47aac4ce7e381 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:04:02.350129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36838974-7319-4b8e-a323-1eb0c5a898a4 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:02.446718 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36838974-7319-4b8e-a323-1eb0c5a898a4 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:02.469412 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:02.479924 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3c8d0-a1f4-4d87-a8d8-2dd441f0021e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:02.480560 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3c8d0-a1f4-4d87-a8d8-2dd441f0021e None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:02.480849 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3c8d0-a1f4-4d87-a8d8-2dd441f0021e None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:02.481128 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3c8d0-a1f4-4d87-a8d8-2dd441f0021e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:02.545839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d3c8d0-a1f4-4d87-a8d8-2dd441f0021e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:02.547904 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d3c8d0-a1f4-4d87-a8d8-2dd441f0021e None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:02.641545 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 575/1128] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:04:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 16:04:02.647234 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c245e2-bb41-4746-a17a-b2348cdc9037 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:02.742313 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c245e2-bb41-4746-a17a-b2348cdc9037 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:02.774447 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8534a0f5-5ad7-4ff1-a2fd-d5a429d9e96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:02.775285 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534a0f5-5ad7-4ff1-a2fd-d5a429d9e96a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:02.775599 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534a0f5-5ad7-4ff1-a2fd-d5a429d9e96a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:02.775878 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534a0f5-5ad7-4ff1-a2fd-d5a429d9e96a None admin] PATH_INFO: `/v3/projects/24b01e54c67341d490411f88e85a39f1` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:02.777042 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8534a0f5-5ad7-4ff1-a2fd-d5a429d9e96a None admin] RBAC: Authorizing `identity:get_project(project_id=24b01e54c67341d490411f88e85a39f1)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:02.778707 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8534a0f5-5ad7-4ff1-a2fd-d5a429d9e96a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:02.780389 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 554/1129] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:04:02 2022] GET /identity/v3/projects/24b01e54c67341d490411f88e85a39f1 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:04:02.786064 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3c8d0-a1f4-4d87-a8d8-2dd441f0021e None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:02.809921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:02.883031 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3c8d0-a1f4-4d87-a8d8-2dd441f0021e None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:02.916211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd932c6-3fb7-4cd2-99e0-464eacc8e238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:02.916850 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd932c6-3fb7-4cd2-99e0-464eacc8e238 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:02.917137 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd932c6-3fb7-4cd2-99e0-464eacc8e238 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:02.917416 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd932c6-3fb7-4cd2-99e0-464eacc8e238 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:02.918242 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd932c6-3fb7-4cd2-99e0-464eacc8e238 None admin] RBAC: Adding query filter params (role.id=2248ac4d70c34e459b8942dec0d7656b, scope.project.id=24b01e54c67341d490411f88e85a39f1, user.id=2976362dbeef430aa4a47aac4ce7e381) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:04:02.918560 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd932c6-3fb7-4cd2-99e0-464eacc8e238 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:02.920009 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd932c6-3fb7-4cd2-99e0-464eacc8e238 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:02.925711 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 576/1130] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:04:02 2022] GET /identity/v3/role_assignments?user.id=2976362dbeef430aa4a47aac4ce7e381&scope.project.id=24b01e54c67341d490411f88e85a39f1&role.id=2248ac4d70c34e459b8942dec0d7656b => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:04:05.130244 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3878ef-3c78-444c-8166-7da9b8d0b21a None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:05.130796 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3878ef-3c78-444c-8166-7da9b8d0b21a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:05.130796 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3878ef-3c78-444c-8166-7da9b8d0b21a None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:05.132243 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 555/1131] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:04:05.137278 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cac34-dc18-4b4f-a3c6-a9ba89e6cd80 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:05.137664 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cac34-dc18-4b4f-a3c6-a9ba89e6cd80 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:05.137830 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cac34-dc18-4b4f-a3c6-a9ba89e6cd80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:05.154879 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-df9cac34-dc18-4b4f-a3c6-a9ba89e6cd80 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:04:05.156116 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-df9cac34-dc18-4b4f-a3c6-a9ba89e6cd80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:04:05.219694 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 577/1132] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:04:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:04:05.242915 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3878ef-3c78-444c-8166-7da9b8d0b21a None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:05.295018 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:05.340484 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3878ef-3c78-444c-8166-7da9b8d0b21a None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:05.374203 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af567433-a20c-4270-888b-6cbf7caaac17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:05.374976 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af567433-a20c-4270-888b-6cbf7caaac17 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:05.375260 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af567433-a20c-4270-888b-6cbf7caaac17 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:05.375567 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af567433-a20c-4270-888b-6cbf7caaac17 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:05.376726 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af567433-a20c-4270-888b-6cbf7caaac17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:05.378144 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af567433-a20c-4270-888b-6cbf7caaac17 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:05.379785 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 556/1133] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:04:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:04:05.385554 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9cac34-dc18-4b4f-a3c6-a9ba89e6cd80 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:05.403348 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:05.481114 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9cac34-dc18-4b4f-a3c6-a9ba89e6cd80 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:05.514036 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b924f-2569-4e63-b2c5-4dc2714f8888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:05.514682 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b924f-2569-4e63-b2c5-4dc2714f8888 None admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:05.515514 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b924f-2569-4e63-b2c5-4dc2714f8888 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:05.515794 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b924f-2569-4e63-b2c5-4dc2714f8888 None admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:05.516492 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7b924f-2569-4e63-b2c5-4dc2714f8888 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:05.517909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7b924f-2569-4e63-b2c5-4dc2714f8888 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:05.542993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 578/1134] 10.222.0.41 () {60 vars in 1141 bytes} [Fri Jun 3 16:04:05 2022] POST /identity/v3/projects => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:04:07.775807 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9d1d0-1f72-4de4-bb09-38a0986c5afc None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:07.776324 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9d1d0-1f72-4de4-bb09-38a0986c5afc None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:07.776416 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9d1d0-1f72-4de4-bb09-38a0986c5afc None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:07.777975 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 557/1135] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:04:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:04:07.783143 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc8c9e-f79d-44d7-a5ad-96e18c46a0c5 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:07.783585 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc8c9e-f79d-44d7-a5ad-96e18c46a0c5 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:07.783731 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc8c9e-f79d-44d7-a5ad-96e18c46a0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:07.801425 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-42dc8c9e-f79d-44d7-a5ad-96e18c46a0c5 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:04:07.802602 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-42dc8c9e-f79d-44d7-a5ad-96e18c46a0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:04:07.862090 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:07.866887 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 579/1136] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:04:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:04:07.892057 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb9d1d0-1f72-4de4-bb09-38a0986c5afc None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:07.991006 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb9d1d0-1f72-4de4-bb09-38a0986c5afc None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:08.010162 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:08.024535 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c0327b-2a0a-48aa-8b22-8837fbd61323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:08.025311 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0327b-2a0a-48aa-8b22-8837fbd61323 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:08.025597 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0327b-2a0a-48aa-8b22-8837fbd61323 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:08.025922 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0327b-2a0a-48aa-8b22-8837fbd61323 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:08.027332 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c0327b-2a0a-48aa-8b22-8837fbd61323 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:08.028760 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c0327b-2a0a-48aa-8b22-8837fbd61323 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:08.031072 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 558/1137] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:04:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:04:08.036516 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc8c9e-f79d-44d7-a5ad-96e18c46a0c5 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:08.131871 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc8c9e-f79d-44d7-a5ad-96e18c46a0c5 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:08.164607 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c1c49-20e5-43d0-9422-47f9ad5a69b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:08.165262 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c1c49-20e5-43d0-9422-47f9ad5a69b0 None admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:08.165550 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c1c49-20e5-43d0-9422-47f9ad5a69b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:08.165829 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c1c49-20e5-43d0-9422-47f9ad5a69b0 None admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:08.166543 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c1c49-20e5-43d0-9422-47f9ad5a69b0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:08.168000 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c1c49-20e5-43d0-9422-47f9ad5a69b0 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:08.187717 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 580/1138] 10.222.0.41 () {60 vars in 1132 bytes} [Fri Jun 3 16:04:08 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:04:10.423414 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c0d35-95d6-436d-8195-f598ca49f387 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:10.423918 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c0d35-95d6-436d-8195-f598ca49f387 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:10.424027 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c0d35-95d6-436d-8195-f598ca49f387 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:10.425331 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 559/1139] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:04:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:04:10.430384 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26083ade-d782-42ee-a91a-3e118257b7df None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:10.430795 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26083ade-d782-42ee-a91a-3e118257b7df None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:10.430937 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26083ade-d782-42ee-a91a-3e118257b7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:10.447654 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-26083ade-d782-42ee-a91a-3e118257b7df None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:04:10.449280 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-26083ade-d782-42ee-a91a-3e118257b7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:04:10.496196 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:10.513044 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 581/1140] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:04:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:04:10.537259 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c0d35-95d6-436d-8195-f598ca49f387 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:10.635260 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c0d35-95d6-436d-8195-f598ca49f387 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:10.668753 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d2942-ebe0-4635-b199-f468e2428981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:10.669525 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d2942-ebe0-4635-b199-f468e2428981 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:10.669810 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d2942-ebe0-4635-b199-f468e2428981 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:10.670086 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d2942-ebe0-4635-b199-f468e2428981 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:10.680048 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680d2942-ebe0-4635-b199-f468e2428981 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:04:10.688161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-680d2942-ebe0-4635-b199-f468e2428981 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:04:10.688161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:04:10.688161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:04:10.688161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:04:10.688161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:04:10.688161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:04:10.688161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:04:10.688161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:04:10.688161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:04:10.688161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:04:10.688161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:04:10.688161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:04:10.688161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:04:10.688161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:04:10.688161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:04:10.688161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:04:10.689259 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:04:10.689259 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:04:10.689259 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:04:10.689259 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:04:10.689259 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:04:10.689259 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:04:10.689259 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:04:10.689259 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:04:10.689259 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:04:10.689259 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:04:10.689259 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:04:10.689259 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:04:10.689259 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:04:10.689259 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:04:10.689259 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:04:10.689259 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:04:10.690328 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:04:10.690328 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:04:10.690328 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:04:10.690328 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:04:10.690328 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:04:10.690328 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:04:10.690328 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:04:10.690328 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:04:10.690328 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:04:10.690328 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:04:10.690328 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:04:10.690328 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:04:10.690328 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 560/1141] 10.222.0.41 () {56 vars in 1096 bytes} [Fri Jun 3 16:04:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:04:10.694992 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26083ade-d782-42ee-a91a-3e118257b7df None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:10.788672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26083ade-d782-42ee-a91a-3e118257b7df None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:10.821427 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf68e8-119e-4b3f-a7ba-24c93f0660f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:10.822073 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf68e8-119e-4b3f-a7ba-24c93f0660f9 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:10.822355 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf68e8-119e-4b3f-a7ba-24c93f0660f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:10.822628 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf68e8-119e-4b3f-a7ba-24c93f0660f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:10.823395 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf68e8-119e-4b3f-a7ba-24c93f0660f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:04:10.823705 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf68e8-119e-4b3f-a7ba-24c93f0660f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:10.825055 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf68e8-119e-4b3f-a7ba-24c93f0660f9 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:10.832736 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 582/1142] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:04:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:04:10.838025 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d2942-ebe0-4635-b199-f468e2428981 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:10.885974 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:10.934879 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d2942-ebe0-4635-b199-f468e2428981 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:10.966741 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68f52c-b672-4c7f-89ee-695673743bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:10.969179 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68f52c-b672-4c7f-89ee-695673743bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:10.969179 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68f52c-b672-4c7f-89ee-695673743bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:10.969179 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68f52c-b672-4c7f-89ee-695673743bb8 None admin] PATH_INFO: `/v3/users/a7760c42e14a45f589b7bbb3ec1003b7` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:10.981962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a68f52c-b672-4c7f-89ee-695673743bb8 None admin] RBAC: Authorizing `identity:get_user(user_id=a7760c42e14a45f589b7bbb3ec1003b7)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:10.983978 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a68f52c-b672-4c7f-89ee-695673743bb8 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:10.985628 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 561/1143] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:04:10 2022] GET /identity/v3/users/a7760c42e14a45f589b7bbb3ec1003b7 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:04:10.991219 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf68e8-119e-4b3f-a7ba-24c93f0660f9 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:11.044732 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:11.094617 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf68e8-119e-4b3f-a7ba-24c93f0660f9 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:11.126275 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6367bb3-9fcf-4529-93d9-c5374528819b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:11.126907 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6367bb3-9fcf-4529-93d9-c5374528819b None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:11.127188 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6367bb3-9fcf-4529-93d9-c5374528819b None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:11.127491 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6367bb3-9fcf-4529-93d9-c5374528819b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:11.191092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6367bb3-9fcf-4529-93d9-c5374528819b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:11.192541 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6367bb3-9fcf-4529-93d9-c5374528819b None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:11.237071 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:11.287409 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 583/1144] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:04:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 16:04:11.293179 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68f52c-b672-4c7f-89ee-695673743bb8 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:11.389162 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68f52c-b672-4c7f-89ee-695673743bb8 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:11.396838 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:11.422895 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a12c9e-67a7-454a-b2cf-3803bf580cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:11.423777 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a12c9e-67a7-454a-b2cf-3803bf580cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:11.424290 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a12c9e-67a7-454a-b2cf-3803bf580cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:11.424374 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a12c9e-67a7-454a-b2cf-3803bf580cf1 None admin] PATH_INFO: `/v3/projects/3ed409fe97c24fdbb650f310a950de1e` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:11.425478 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a12c9e-67a7-454a-b2cf-3803bf580cf1 None admin] RBAC: Authorizing `identity:get_project(project_id=3ed409fe97c24fdbb650f310a950de1e)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:11.426906 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a12c9e-67a7-454a-b2cf-3803bf580cf1 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:11.428623 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 562/1145] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:04:11 2022] GET /identity/v3/projects/3ed409fe97c24fdbb650f310a950de1e => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:04:11.434776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6367bb3-9fcf-4529-93d9-c5374528819b None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:11.529442 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6367bb3-9fcf-4529-93d9-c5374528819b None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:11.562309 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f12ec9-3966-41d2-b490-8e77c2a571b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:11.562946 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f12ec9-3966-41d2-b490-8e77c2a571b1 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:11.563229 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f12ec9-3966-41d2-b490-8e77c2a571b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:11.563531 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f12ec9-3966-41d2-b490-8e77c2a571b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:11.564347 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f12ec9-3966-41d2-b490-8e77c2a571b1 None admin] RBAC: Adding query filter params (role.id=75936a608b8149099ff54e317f3a6740, scope.project.id=3ed409fe97c24fdbb650f310a950de1e, user.id=a7760c42e14a45f589b7bbb3ec1003b7) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:04:11.564657 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f12ec9-3966-41d2-b490-8e77c2a571b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:11.566059 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f12ec9-3966-41d2-b490-8e77c2a571b1 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:11.571282 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 584/1146] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:04:11 2022] GET /identity/v3/role_assignments?user.id=a7760c42e14a45f589b7bbb3ec1003b7&scope.project.id=3ed409fe97c24fdbb650f310a950de1e&role.id=75936a608b8149099ff54e317f3a6740 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:04:13.744664 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60790f3-a2c5-4879-b1a9-4ce6a9ef091c None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:13.745213 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60790f3-a2c5-4879-b1a9-4ce6a9ef091c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:13.745213 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60790f3-a2c5-4879-b1a9-4ce6a9ef091c None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:13.746566 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 563/1147] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:04:13.751705 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004cc0f-950a-4739-b07c-621e309598d7 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:13.752114 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004cc0f-950a-4739-b07c-621e309598d7 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:13.752258 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004cc0f-950a-4739-b07c-621e309598d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:13.769010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-a004cc0f-950a-4739-b07c-621e309598d7 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:04:13.770182 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-a004cc0f-950a-4739-b07c-621e309598d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:04:13.777466 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:13.834562 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 585/1148] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:04:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:04:13.857411 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60790f3-a2c5-4879-b1a9-4ce6a9ef091c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:13.955609 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60790f3-a2c5-4879-b1a9-4ce6a9ef091c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:13.988528 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f880344-06b0-4f1c-9548-dc5dc2999907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:13.989300 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f880344-06b0-4f1c-9548-dc5dc2999907 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:13.989584 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f880344-06b0-4f1c-9548-dc5dc2999907 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:13.989857 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f880344-06b0-4f1c-9548-dc5dc2999907 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:13.999768 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f880344-06b0-4f1c-9548-dc5dc2999907 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:04:14.007150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-9f880344-06b0-4f1c-9548-dc5dc2999907 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:04:14.007150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:04:14.007150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:04:14.007150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:04:14.007150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:04:14.007150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:04:14.007150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:04:14.007150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:04:14.007150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:04:14.007150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:04:14.007150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:04:14.007150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:04:14.007150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:04:14.007150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:04:14.007150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:04:14.007150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:04:14.008306 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:04:14.008306 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:04:14.008306 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:04:14.008306 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:04:14.008306 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:04:14.008306 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:04:14.008306 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:04:14.008306 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:04:14.008306 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:04:14.008306 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:04:14.008306 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:04:14.008306 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:04:14.008306 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:04:14.008306 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:04:14.008306 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:04:14.008306 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:04:14.009206 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:04:14.009206 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:04:14.009206 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:04:14.009206 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:04:14.009206 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:04:14.009206 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:04:14.009206 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:04:14.009206 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:04:14.009206 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:04:14.009206 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:04:14.009206 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:04:14.009206 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:04:14.009206 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 564/1149] 10.222.0.41 () {56 vars in 1096 bytes} [Fri Jun 3 16:04:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:04:14.014929 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a004cc0f-950a-4739-b07c-621e309598d7 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:14.109607 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a004cc0f-950a-4739-b07c-621e309598d7 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:14.142532 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24949e4-0bfb-4851-9018-c05d23e5c44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:14.143183 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24949e4-0bfb-4851-9018-c05d23e5c44b None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:14.143500 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24949e4-0bfb-4851-9018-c05d23e5c44b None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:14.143785 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24949e4-0bfb-4851-9018-c05d23e5c44b None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:14.144522 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24949e4-0bfb-4851-9018-c05d23e5c44b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:04:14.144833 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24949e4-0bfb-4851-9018-c05d23e5c44b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:14.146193 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24949e4-0bfb-4851-9018-c05d23e5c44b None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:14.154128 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 586/1150] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:04:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:04:14.159720 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f880344-06b0-4f1c-9548-dc5dc2999907 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:14.193128 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:14.256170 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f880344-06b0-4f1c-9548-dc5dc2999907 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:14.288309 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce48962-9f7f-4984-b5b4-cf0c9bcb81c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:14.289161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce48962-9f7f-4984-b5b4-cf0c9bcb81c7 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:14.289445 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce48962-9f7f-4984-b5b4-cf0c9bcb81c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:14.289719 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce48962-9f7f-4984-b5b4-cf0c9bcb81c7 None admin] PATH_INFO: `/v3/users/a7760c42e14a45f589b7bbb3ec1003b7` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:14.290892 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce48962-9f7f-4984-b5b4-cf0c9bcb81c7 None admin] RBAC: Authorizing `identity:get_user(user_id=a7760c42e14a45f589b7bbb3ec1003b7)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:14.292345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce48962-9f7f-4984-b5b4-cf0c9bcb81c7 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:14.293943 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 565/1151] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:04:14 2022] GET /identity/v3/users/a7760c42e14a45f589b7bbb3ec1003b7 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:04:14.299577 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24949e4-0bfb-4851-9018-c05d23e5c44b None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:14.303266 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:14.397285 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24949e4-0bfb-4851-9018-c05d23e5c44b None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:14.431220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4746da-ee4d-4b29-aba7-93be47958abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:14.432106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4746da-ee4d-4b29-aba7-93be47958abc None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:14.432394 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4746da-ee4d-4b29-aba7-93be47958abc None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:14.432675 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4746da-ee4d-4b29-aba7-93be47958abc None admin] PATH_INFO: `/v3/projects/3ed409fe97c24fdbb650f310a950de1e` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:14.433848 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4746da-ee4d-4b29-aba7-93be47958abc None admin] RBAC: Authorizing `identity:get_project(project_id=3ed409fe97c24fdbb650f310a950de1e)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:14.435264 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4746da-ee4d-4b29-aba7-93be47958abc None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:14.437009 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 587/1152] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:04:14 2022] GET /identity/v3/projects/3ed409fe97c24fdbb650f310a950de1e => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:04:14.443008 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce48962-9f7f-4984-b5b4-cf0c9bcb81c7 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:14.540084 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce48962-9f7f-4984-b5b4-cf0c9bcb81c7 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:14.541465 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:14.572713 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867ca05e-5e7e-4b5b-82bf-a73740b6a803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:14.573584 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ca05e-5e7e-4b5b-82bf-a73740b6a803 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:14.573865 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ca05e-5e7e-4b5b-82bf-a73740b6a803 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:14.574139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ca05e-5e7e-4b5b-82bf-a73740b6a803 None admin] PATH_INFO: `/v3/projects/3ed409fe97c24fdbb650f310a950de1e/users/a7760c42e14a45f589b7bbb3ec1003b7/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:14.576102 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867ca05e-5e7e-4b5b-82bf-a73740b6a803 None admin] RBAC: Authorizing `identity:create_grant(project_id=3ed409fe97c24fdbb650f310a950de1e, user_id=a7760c42e14a45f589b7bbb3ec1003b7, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:14.578035 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867ca05e-5e7e-4b5b-82bf-a73740b6a803 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:14.590316 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 566/1153] 10.222.0.41 () {58 vars in 1439 bytes} [Fri Jun 3 16:04:14 2022] PUT /identity/v3/projects/3ed409fe97c24fdbb650f310a950de1e/users/a7760c42e14a45f589b7bbb3ec1003b7/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:04:16.804258 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9ca46-06a8-4875-9fe1-8bda647e5497 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:16.804807 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9ca46-06a8-4875-9fe1-8bda647e5497 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:16.804900 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9ca46-06a8-4875-9fe1-8bda647e5497 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:16.806198 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 588/1154] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:04:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:04:16.811355 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a575c68-31d1-4f70-8e6f-77771e3b6fd0 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:16.811717 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a575c68-31d1-4f70-8e6f-77771e3b6fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:16.811922 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a575c68-31d1-4f70-8e6f-77771e3b6fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:16.828722 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-7a575c68-31d1-4f70-8e6f-77771e3b6fd0 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:04:16.829913 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7a575c68-31d1-4f70-8e6f-77771e3b6fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:04:16.892879 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 567/1155] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:04:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:04:16.915987 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c9ca46-06a8-4875-9fe1-8bda647e5497 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:16.984651 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:17.014597 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c9ca46-06a8-4875-9fe1-8bda647e5497 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:17.054772 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e88762-e42b-4e80-ad13-d573b791193a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:17.055508 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e88762-e42b-4e80-ad13-d573b791193a None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:17.055791 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e88762-e42b-4e80-ad13-d573b791193a None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:17.056064 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e88762-e42b-4e80-ad13-d573b791193a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:17.066499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e88762-e42b-4e80-ad13-d573b791193a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:04:17.074107 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-e8e88762-e42b-4e80-ad13-d573b791193a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:04:17.074107 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:04:17.074107 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:04:17.074107 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:04:17.074107 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:04:17.074107 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:04:17.074107 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:04:17.074107 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:04:17.074107 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:04:17.074107 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:04:17.074107 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:04:17.074107 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:04:17.074107 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:04:17.074107 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:04:17.074107 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:04:17.074107 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:04:17.075164 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:04:17.075164 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:04:17.075164 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:04:17.075164 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:04:17.075164 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:04:17.075164 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:04:17.075164 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:04:17.075164 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:04:17.075164 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:04:17.075164 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:04:17.075164 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:04:17.075164 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:04:17.075164 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:04:17.075164 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:04:17.075164 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:04:17.075164 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:04:17.076076 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:04:17.076076 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:04:17.076076 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:04:17.076076 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:04:17.076076 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:04:17.076076 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:04:17.076076 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:04:17.076076 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:04:17.076076 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:04:17.076076 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:04:17.076076 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:04:17.076076 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:04:17.076076 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 589/1156] 10.222.0.41 () {56 vars in 1096 bytes} [Fri Jun 3 16:04:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:04:17.081335 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a575c68-31d1-4f70-8e6f-77771e3b6fd0 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:17.169431 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:17.179092 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a575c68-31d1-4f70-8e6f-77771e3b6fd0 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:17.210816 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480c55f6-0a0b-42b2-8881-c7a3ae24d5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:17.211496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c55f6-0a0b-42b2-8881-c7a3ae24d5ac None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:17.211782 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c55f6-0a0b-42b2-8881-c7a3ae24d5ac None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:17.212059 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c55f6-0a0b-42b2-8881-c7a3ae24d5ac None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:17.212804 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480c55f6-0a0b-42b2-8881-c7a3ae24d5ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:04:17.213110 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480c55f6-0a0b-42b2-8881-c7a3ae24d5ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:17.214472 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480c55f6-0a0b-42b2-8881-c7a3ae24d5ac None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:17.222748 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 568/1157] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:04:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:04:17.228638 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e88762-e42b-4e80-ad13-d573b791193a None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:17.325626 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e88762-e42b-4e80-ad13-d573b791193a None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:17.353185 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:17.359063 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d224e78-2597-4f56-9c9e-381862863c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:17.360061 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d224e78-2597-4f56-9c9e-381862863c5e None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:17.360340 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d224e78-2597-4f56-9c9e-381862863c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:17.360606 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d224e78-2597-4f56-9c9e-381862863c5e None admin] PATH_INFO: `/v3/users/a7760c42e14a45f589b7bbb3ec1003b7` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:17.361770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d224e78-2597-4f56-9c9e-381862863c5e None admin] RBAC: Authorizing `identity:get_user(user_id=a7760c42e14a45f589b7bbb3ec1003b7)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:17.363169 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d224e78-2597-4f56-9c9e-381862863c5e None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:17.364790 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 590/1158] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:04:17 2022] GET /identity/v3/users/a7760c42e14a45f589b7bbb3ec1003b7 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:04:17.370503 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480c55f6-0a0b-42b2-8881-c7a3ae24d5ac None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:17.475027 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480c55f6-0a0b-42b2-8881-c7a3ae24d5ac None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:17.506444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0df367-21c0-435e-add1-ce4a7933fec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:17.507079 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0df367-21c0-435e-add1-ce4a7933fec3 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:17.507394 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0df367-21c0-435e-add1-ce4a7933fec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:17.507675 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0df367-21c0-435e-add1-ce4a7933fec3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:17.531391 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:17.572479 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0df367-21c0-435e-add1-ce4a7933fec3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:17.573917 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0df367-21c0-435e-add1-ce4a7933fec3 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:17.667898 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 569/1159] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:04:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 16:04:17.673805 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d224e78-2597-4f56-9c9e-381862863c5e None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:17.769135 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d224e78-2597-4f56-9c9e-381862863c5e None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:17.801774 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3d2a74-5039-48be-9169-6c319abcfec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:17.802623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d2a74-5039-48be-9169-6c319abcfec5 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:17.802902 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d2a74-5039-48be-9169-6c319abcfec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:17.803174 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d2a74-5039-48be-9169-6c319abcfec5 None admin] PATH_INFO: `/v3/projects/3ed409fe97c24fdbb650f310a950de1e` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:17.804352 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3d2a74-5039-48be-9169-6c319abcfec5 None admin] RBAC: Authorizing `identity:get_project(project_id=3ed409fe97c24fdbb650f310a950de1e)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:17.805763 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3d2a74-5039-48be-9169-6c319abcfec5 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:17.807463 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 591/1160] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:04:17 2022] GET /identity/v3/projects/3ed409fe97c24fdbb650f310a950de1e => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:04:17.813675 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0df367-21c0-435e-add1-ce4a7933fec3 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:17.866380 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:17.909296 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0df367-21c0-435e-add1-ce4a7933fec3 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:17.941572 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a1427-d2d7-4ecd-84db-cfa4372c75fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:17.942148 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a1427-d2d7-4ecd-84db-cfa4372c75fb None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:17.942432 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a1427-d2d7-4ecd-84db-cfa4372c75fb None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:17.942708 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a1427-d2d7-4ecd-84db-cfa4372c75fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:17.943554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83a1427-d2d7-4ecd-84db-cfa4372c75fb None admin] RBAC: Adding query filter params (role.id=75936a608b8149099ff54e317f3a6740, scope.project.id=3ed409fe97c24fdbb650f310a950de1e, user.id=a7760c42e14a45f589b7bbb3ec1003b7) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:04:17.943866 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83a1427-d2d7-4ecd-84db-cfa4372c75fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:17.945283 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83a1427-d2d7-4ecd-84db-cfa4372c75fb None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:17.950920 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 570/1161] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:04:17 2022] GET /identity/v3/role_assignments?user.id=a7760c42e14a45f589b7bbb3ec1003b7&scope.project.id=3ed409fe97c24fdbb650f310a950de1e&role.id=75936a608b8149099ff54e317f3a6740 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:04:20.175704 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf9d77-7907-475c-a079-51d172c17012 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:20.176293 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf9d77-7907-475c-a079-51d172c17012 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:20.176293 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf9d77-7907-475c-a079-51d172c17012 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:20.177604 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 592/1162] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:04:20.182959 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b1ec9-611b-48ad-8e4b-9ccc05eb53cf None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:20.183447 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b1ec9-611b-48ad-8e4b-9ccc05eb53cf None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:20.183549 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b1ec9-611b-48ad-8e4b-9ccc05eb53cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:20.199792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b72b1ec9-611b-48ad-8e4b-9ccc05eb53cf None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:04:20.200976 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b72b1ec9-611b-48ad-8e4b-9ccc05eb53cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:04:20.264580 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 571/1163] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:04:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:04:20.287509 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf9d77-7907-475c-a079-51d172c17012 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:20.354774 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:20.387647 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf9d77-7907-475c-a079-51d172c17012 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:20.420016 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9ad6e-d2a8-4eb4-88be-9edaf8728dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:20.421242 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9ad6e-d2a8-4eb4-88be-9edaf8728dfe None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:20.421525 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9ad6e-d2a8-4eb4-88be-9edaf8728dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:20.421803 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9ad6e-d2a8-4eb4-88be-9edaf8728dfe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:20.485589 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a9ad6e-d2a8-4eb4-88be-9edaf8728dfe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:20.486956 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a9ad6e-d2a8-4eb4-88be-9edaf8728dfe None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:20.549166 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:20.581596 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 593/1164] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:04:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 16:04:20.587549 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72b1ec9-611b-48ad-8e4b-9ccc05eb53cf None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:20.633554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:20.681848 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72b1ec9-611b-48ad-8e4b-9ccc05eb53cf None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:20.713831 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9469fd45-c624-4385-a5a3-9821a4e6b968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:20.714602 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469fd45-c624-4385-a5a3-9821a4e6b968 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:20.714882 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469fd45-c624-4385-a5a3-9821a4e6b968 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:20.715153 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469fd45-c624-4385-a5a3-9821a4e6b968 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:20.722116 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9469fd45-c624-4385-a5a3-9821a4e6b968 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:20.723595 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9469fd45-c624-4385-a5a3-9821a4e6b968 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:20.737249 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-9469fd45-c624-4385-a5a3-9821a4e6b968 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 03 16:04:20.737249 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:04:20.737249 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 03 16:04:20.737249 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 03 16:04:20.737249 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:04:20.737249 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:04:20.737249 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:04:20.737249 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:04:20.737249 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 03 16:04:20.737249 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:04:20.737249 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 16:04:20.737249 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:04:20.737249 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:04:20.737249 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:04:20.737249 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:04:20.737249 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:04:20.737249 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:04:20.738474 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:04:20.738474 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:04:20.738474 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:04:20.738474 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:04:20.738474 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:04:20.738474 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:04:20.738474 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:04:20.738474 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:04:20.738474 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:04:20.738474 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:04:20.738474 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:04:20.738474 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:04:20.738474 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:04:20.738474 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:04:20.738474 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:04:20.738474 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:04:20.739633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:04:20.739633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:04:20.739633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:04:20.739633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:04:20.739633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:04:20.739633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:04:20.739633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:04:20.739633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:04:20.739633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:04:20.739633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:04:20.739633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 03 16:04:20.739633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:04:20.739633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 03 16:04:20.739633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:04:20.739633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 572/1165] 10.222.0.41 () {56 vars in 1117 bytes} [Fri Jun 3 16:04:20 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:04:20.744093 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9ad6e-d2a8-4eb4-88be-9edaf8728dfe None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:20.837998 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9ad6e-d2a8-4eb4-88be-9edaf8728dfe None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:20.870362 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c3d9f0-b0ab-46fa-9d8e-6149510df4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:20.871006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c3d9f0-b0ab-46fa-9d8e-6149510df4eb None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:20.871315 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c3d9f0-b0ab-46fa-9d8e-6149510df4eb None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:20.871595 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c3d9f0-b0ab-46fa-9d8e-6149510df4eb None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:20.872307 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c3d9f0-b0ab-46fa-9d8e-6149510df4eb None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:04:20.872616 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c3d9f0-b0ab-46fa-9d8e-6149510df4eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:20.874497 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c3d9f0-b0ab-46fa-9d8e-6149510df4eb None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:20.881349 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 594/1166] 10.222.0.41 () {56 vars in 1115 bytes} [Fri Jun 3 16:04:20 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:04:23.052648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21107787-3527-48c5-8b47-c99105ec68e1 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:23.053191 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21107787-3527-48c5-8b47-c99105ec68e1 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:23.053191 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21107787-3527-48c5-8b47-c99105ec68e1 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:23.054565 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 573/1167] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:04:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:04:23.059624 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2aa908-9ca6-4a16-b81d-03bd55482c4a None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:23.060073 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2aa908-9ca6-4a16-b81d-03bd55482c4a None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:23.060174 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2aa908-9ca6-4a16-b81d-03bd55482c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:23.077694 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-dc2aa908-9ca6-4a16-b81d-03bd55482c4a None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:04:23.078868 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-dc2aa908-9ca6-4a16-b81d-03bd55482c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:04:23.104119 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:23.143963 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 595/1168] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:04:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:04:23.167381 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21107787-3527-48c5-8b47-c99105ec68e1 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:23.265309 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21107787-3527-48c5-8b47-c99105ec68e1 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:23.273654 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:23.298488 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f10255-f52a-40d6-9ebc-ddbb7c4a5045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:23.299130 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f10255-f52a-40d6-9ebc-ddbb7c4a5045 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:23.299449 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f10255-f52a-40d6-9ebc-ddbb7c4a5045 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:23.299722 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f10255-f52a-40d6-9ebc-ddbb7c4a5045 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:23.300451 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f10255-f52a-40d6-9ebc-ddbb7c4a5045 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:23.301799 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f10255-f52a-40d6-9ebc-ddbb7c4a5045 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:23.315232 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 574/1169] 10.222.0.41 () {60 vars in 1138 bytes} [Fri Jun 3 16:04:23 2022] POST /identity/v3/domains => generated 267 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:04:25.541379 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457faa1-a7cb-4c5c-8899-47d60f880efb None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:25.541895 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457faa1-a7cb-4c5c-8899-47d60f880efb None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:25.541895 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457faa1-a7cb-4c5c-8899-47d60f880efb None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:25.543208 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 596/1170] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:04:25.549008 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411c0b7-6f1c-487c-9e54-687ceb68a25e None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:25.549388 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411c0b7-6f1c-487c-9e54-687ceb68a25e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:25.549565 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411c0b7-6f1c-487c-9e54-687ceb68a25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:25.566317 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b411c0b7-6f1c-487c-9e54-687ceb68a25e None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:04:25.567520 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b411c0b7-6f1c-487c-9e54-687ceb68a25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:04:25.630624 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 575/1171] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:04:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:04:25.653821 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0457faa1-a7cb-4c5c-8899-47d60f880efb None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:25.751229 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0457faa1-a7cb-4c5c-8899-47d60f880efb None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:25.784237 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf4893e-6cf5-4032-a763-d1af82568511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:25.785072 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf4893e-6cf5-4032-a763-d1af82568511 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:25.785353 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf4893e-6cf5-4032-a763-d1af82568511 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:25.785624 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf4893e-6cf5-4032-a763-d1af82568511 None admin] PATH_INFO: `/v3/domains/d786ef3cc65f42a6b4c6e936a404e3b7` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:25.795646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf4893e-6cf5-4032-a763-d1af82568511 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d786ef3cc65f42a6b4c6e936a404e3b7)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:25.797024 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf4893e-6cf5-4032-a763-d1af82568511 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:25.798607 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 597/1172] 10.222.0.41 () {56 vars in 1183 bytes} [Fri Jun 3 16:04:25 2022] GET /identity/v3/domains/d786ef3cc65f42a6b4c6e936a404e3b7 => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:04:25.804173 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b411c0b7-6f1c-487c-9e54-687ceb68a25e None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:25.895023 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:25.899188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b411c0b7-6f1c-487c-9e54-687ceb68a25e None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:25.933578 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9221c51f-cde1-43c0-a2af-0d772a50bcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:25.934226 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221c51f-cde1-43c0-a2af-0d772a50bcc1 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:25.934512 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221c51f-cde1-43c0-a2af-0d772a50bcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:25.934793 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221c51f-cde1-43c0-a2af-0d772a50bcc1 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:25.935537 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9221c51f-cde1-43c0-a2af-0d772a50bcc1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:25.936987 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9221c51f-cde1-43c0-a2af-0d772a50bcc1 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:25.960717 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 576/1173] 10.222.0.41 () {60 vars in 1141 bytes} [Fri Jun 3 16:04:25 2022] POST /identity/v3/projects => generated 361 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:04:28.228351 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67b9e8-40ce-447e-a490-9300028336a0 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:28.228886 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67b9e8-40ce-447e-a490-9300028336a0 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:28.228886 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67b9e8-40ce-447e-a490-9300028336a0 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:28.230244 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 598/1174] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:04:28.235538 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1c43d-0ee0-4158-82ba-cb171cd73eab None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:28.235933 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1c43d-0ee0-4158-82ba-cb171cd73eab None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:28.236103 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1c43d-0ee0-4158-82ba-cb171cd73eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:28.254493 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-4aa1c43d-0ee0-4158-82ba-cb171cd73eab None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:04:28.255718 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4aa1c43d-0ee0-4158-82ba-cb171cd73eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:04:28.320031 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 577/1175] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:04:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:04:28.346025 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67b9e8-40ce-447e-a490-9300028336a0 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:28.369990 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:28.444667 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67b9e8-40ce-447e-a490-9300028336a0 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:28.477026 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ec554-b728-4ef7-9a16-e285f00ed45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:28.477875 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ec554-b728-4ef7-9a16-e285f00ed45c None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:28.478158 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ec554-b728-4ef7-9a16-e285f00ed45c None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:28.478480 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ec554-b728-4ef7-9a16-e285f00ed45c None admin] PATH_INFO: `/v3/domains/d786ef3cc65f42a6b4c6e936a404e3b7` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:28.479621 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ec554-b728-4ef7-9a16-e285f00ed45c None admin] RBAC: Authorizing `identity:get_domain(domain_id=d786ef3cc65f42a6b4c6e936a404e3b7)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:28.481044 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ec554-b728-4ef7-9a16-e285f00ed45c None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:28.482663 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 599/1176] 10.222.0.41 () {56 vars in 1183 bytes} [Fri Jun 3 16:04:28 2022] GET /identity/v3/domains/d786ef3cc65f42a6b4c6e936a404e3b7 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:04:28.488252 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1c43d-0ee0-4158-82ba-cb171cd73eab None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:28.566187 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:28.585662 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1c43d-0ee0-4158-82ba-cb171cd73eab None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:28.619662 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb4a27-0eaf-41f8-a56b-3de3f2b2e2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:28.620320 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb4a27-0eaf-41f8-a56b-3de3f2b2e2b7 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:28.620686 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb4a27-0eaf-41f8-a56b-3de3f2b2e2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:28.620974 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb4a27-0eaf-41f8-a56b-3de3f2b2e2b7 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:28.621684 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afb4a27-0eaf-41f8-a56b-3de3f2b2e2b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:28.623163 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afb4a27-0eaf-41f8-a56b-3de3f2b2e2b7 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:28.643281 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 578/1177] 10.222.0.41 () {60 vars in 1132 bytes} [Fri Jun 3 16:04:28 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:04:30.875714 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4681fe-aa32-4317-94f4-e1057fc97a40 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:30.876262 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4681fe-aa32-4317-94f4-e1057fc97a40 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:30.876262 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4681fe-aa32-4317-94f4-e1057fc97a40 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:30.877626 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 600/1178] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:04:30.882704 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef96c25d-f64b-42db-a292-cc03ffda75e9 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:30.883136 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef96c25d-f64b-42db-a292-cc03ffda75e9 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:30.883263 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef96c25d-f64b-42db-a292-cc03ffda75e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:30.899442 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-ef96c25d-f64b-42db-a292-cc03ffda75e9 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:04:30.900614 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ef96c25d-f64b-42db-a292-cc03ffda75e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:04:30.964199 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 579/1179] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:04:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:04:30.987813 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4681fe-aa32-4317-94f4-e1057fc97a40 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:31.076763 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:31.087089 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4681fe-aa32-4317-94f4-e1057fc97a40 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:31.119634 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f533c5aa-c49a-4206-92b9-d28f29049ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:31.120408 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f533c5aa-c49a-4206-92b9-d28f29049ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:31.120695 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f533c5aa-c49a-4206-92b9-d28f29049ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:31.120974 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f533c5aa-c49a-4206-92b9-d28f29049ae4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:31.131201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f533c5aa-c49a-4206-92b9-d28f29049ae4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:04:31.138770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-f533c5aa-c49a-4206-92b9-d28f29049ae4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:04:31.138770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:04:31.138770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:04:31.138770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:04:31.138770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:04:31.138770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:04:31.138770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:04:31.138770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:04:31.138770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:04:31.138770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:04:31.138770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:04:31.138770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:04:31.138770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:04:31.138770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:04:31.138770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:04:31.138770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:04:31.139938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:04:31.139938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:04:31.139938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:04:31.139938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:04:31.139938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:04:31.139938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:04:31.139938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:04:31.139938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:04:31.139938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:04:31.139938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:04:31.139938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:04:31.139938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:04:31.139938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:04:31.139938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:04:31.139938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:04:31.139938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:04:31.140983 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:04:31.140983 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:04:31.140983 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:04:31.140983 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:04:31.140983 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:04:31.140983 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:04:31.140983 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:04:31.140983 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:04:31.140983 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:04:31.140983 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:04:31.140983 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:04:31.140983 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:04:31.140983 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 601/1180] 10.222.0.41 () {56 vars in 1096 bytes} [Fri Jun 3 16:04:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:04:31.145897 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef96c25d-f64b-42db-a292-cc03ffda75e9 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:31.207450 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:31.240638 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef96c25d-f64b-42db-a292-cc03ffda75e9 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:31.273463 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7af5b-f7e5-4f29-9688-fd570ccbf65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:31.274119 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7af5b-f7e5-4f29-9688-fd570ccbf65e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:31.274403 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7af5b-f7e5-4f29-9688-fd570ccbf65e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:31.274682 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7af5b-f7e5-4f29-9688-fd570ccbf65e None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:31.275461 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f7af5b-f7e5-4f29-9688-fd570ccbf65e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:04:31.275774 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f7af5b-f7e5-4f29-9688-fd570ccbf65e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:31.277414 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f7af5b-f7e5-4f29-9688-fd570ccbf65e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:31.285273 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 580/1181] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:04:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:04:31.290946 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f533c5aa-c49a-4206-92b9-d28f29049ae4 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:31.386682 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f533c5aa-c49a-4206-92b9-d28f29049ae4 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:31.418786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533020ff-59f7-4f51-9f2f-11230af9b5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:31.419709 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533020ff-59f7-4f51-9f2f-11230af9b5da None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:31.420490 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533020ff-59f7-4f51-9f2f-11230af9b5da None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:31.420763 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533020ff-59f7-4f51-9f2f-11230af9b5da None admin] PATH_INFO: `/v3/users/7031e51b30dd45d6bf84bfa4dc04ab86` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:31.433243 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533020ff-59f7-4f51-9f2f-11230af9b5da None admin] RBAC: Authorizing `identity:get_user(user_id=7031e51b30dd45d6bf84bfa4dc04ab86)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:31.434659 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533020ff-59f7-4f51-9f2f-11230af9b5da None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:31.436319 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 602/1182] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:04:31 2022] GET /identity/v3/users/7031e51b30dd45d6bf84bfa4dc04ab86 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:04:31.441828 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7af5b-f7e5-4f29-9688-fd570ccbf65e None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:31.537968 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7af5b-f7e5-4f29-9688-fd570ccbf65e None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:31.562803 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:31.570664 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e06e32-03c5-4069-9443-122bc55f1455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:31.571287 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e06e32-03c5-4069-9443-122bc55f1455 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:31.571598 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e06e32-03c5-4069-9443-122bc55f1455 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:31.571875 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e06e32-03c5-4069-9443-122bc55f1455 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:31.635232 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e06e32-03c5-4069-9443-122bc55f1455 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:31.636765 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e06e32-03c5-4069-9443-122bc55f1455 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:31.731117 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 581/1183] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:04:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 16:04:31.736826 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533020ff-59f7-4f51-9f2f-11230af9b5da None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:31.747099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:31.832435 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533020ff-59f7-4f51-9f2f-11230af9b5da None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:31.865309 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d00c48-74df-4721-a0de-1bbf11a118d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:31.866473 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d00c48-74df-4721-a0de-1bbf11a118d8 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:31.866854 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d00c48-74df-4721-a0de-1bbf11a118d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:31.867246 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d00c48-74df-4721-a0de-1bbf11a118d8 None admin] PATH_INFO: `/v3/projects/688971680e16429eb655e00d8fbe95fa` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:31.868926 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d00c48-74df-4721-a0de-1bbf11a118d8 None admin] RBAC: Authorizing `identity:get_project(project_id=688971680e16429eb655e00d8fbe95fa)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:31.871015 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d00c48-74df-4721-a0de-1bbf11a118d8 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:31.872916 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 603/1184] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:04:31 2022] GET /identity/v3/projects/688971680e16429eb655e00d8fbe95fa => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:04:31.878978 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e06e32-03c5-4069-9443-122bc55f1455 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:31.906667 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:31.974449 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e06e32-03c5-4069-9443-122bc55f1455 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:32.007504 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a5d83a-3e12-47a1-8301-6c13df1f6f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:32.008138 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a5d83a-3e12-47a1-8301-6c13df1f6f6d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:32.008421 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a5d83a-3e12-47a1-8301-6c13df1f6f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:32.008694 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a5d83a-3e12-47a1-8301-6c13df1f6f6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:32.009517 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a5d83a-3e12-47a1-8301-6c13df1f6f6d None admin] RBAC: Adding query filter params (role.id=75936a608b8149099ff54e317f3a6740, scope.project.id=688971680e16429eb655e00d8fbe95fa, user.id=7031e51b30dd45d6bf84bfa4dc04ab86) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:04:32.009829 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a5d83a-3e12-47a1-8301-6c13df1f6f6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:32.011247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a5d83a-3e12-47a1-8301-6c13df1f6f6d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:32.016528 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 582/1185] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:04:31 2022] GET /identity/v3/role_assignments?user.id=7031e51b30dd45d6bf84bfa4dc04ab86&scope.project.id=688971680e16429eb655e00d8fbe95fa&role.id=75936a608b8149099ff54e317f3a6740 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:04:34.195730 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a57e0-ffa3-408a-b825-af569de876a8 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:34.196287 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a57e0-ffa3-408a-b825-af569de876a8 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:34.196287 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a57e0-ffa3-408a-b825-af569de876a8 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:34.197632 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 604/1186] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:04:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:04:34.203346 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18eb956-f4e5-45f9-b10d-639ca4d332f5 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:34.203730 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18eb956-f4e5-45f9-b10d-639ca4d332f5 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:34.203910 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18eb956-f4e5-45f9-b10d-639ca4d332f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:34.220696 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-f18eb956-f4e5-45f9-b10d-639ca4d332f5 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:04:34.221871 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f18eb956-f4e5-45f9-b10d-639ca4d332f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:04:34.285662 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 583/1187] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:04:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:04:34.308465 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a57e0-ffa3-408a-b825-af569de876a8 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:34.383538 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:34.407420 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a57e0-ffa3-408a-b825-af569de876a8 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:34.440328 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da8a2bf-f91e-459f-91eb-70cc84ba3226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:34.441099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da8a2bf-f91e-459f-91eb-70cc84ba3226 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:34.441378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da8a2bf-f91e-459f-91eb-70cc84ba3226 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:34.441651 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da8a2bf-f91e-459f-91eb-70cc84ba3226 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:34.451911 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da8a2bf-f91e-459f-91eb-70cc84ba3226 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:04:34.459378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-4da8a2bf-f91e-459f-91eb-70cc84ba3226 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:04:34.459378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:04:34.459378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:04:34.459378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:04:34.459378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:04:34.459378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:04:34.459378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:04:34.459378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:04:34.459378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:04:34.459378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:04:34.459378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:04:34.459378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:04:34.459378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:04:34.459378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:04:34.459378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:04:34.459378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:04:34.460456 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:04:34.460456 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:04:34.460456 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:04:34.460456 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:04:34.460456 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:04:34.460456 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:04:34.460456 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:04:34.460456 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:04:34.460456 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:04:34.460456 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:04:34.460456 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:04:34.460456 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:04:34.460456 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:04:34.460456 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:04:34.460456 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:04:34.460456 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:04:34.461603 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:04:34.461603 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:04:34.461603 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:04:34.461603 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:04:34.461603 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:04:34.461603 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:04:34.461603 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:04:34.461603 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:04:34.461603 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:04:34.461603 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:04:34.461603 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:04:34.461603 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:04:34.461603 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 605/1188] 10.222.0.41 () {56 vars in 1096 bytes} [Fri Jun 3 16:04:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:04:34.466382 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18eb956-f4e5-45f9-b10d-639ca4d332f5 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:34.489202 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:34.562105 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18eb956-f4e5-45f9-b10d-639ca4d332f5 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:34.594581 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b04aa28-2d0d-4b5c-b000-1644093eae74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:34.595237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04aa28-2d0d-4b5c-b000-1644093eae74 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:34.595555 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04aa28-2d0d-4b5c-b000-1644093eae74 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:34.595837 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04aa28-2d0d-4b5c-b000-1644093eae74 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:34.596583 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b04aa28-2d0d-4b5c-b000-1644093eae74 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:04:34.596893 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b04aa28-2d0d-4b5c-b000-1644093eae74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:34.598265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b04aa28-2d0d-4b5c-b000-1644093eae74 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:34.606044 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 584/1189] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:04:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:04:34.611752 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da8a2bf-f91e-459f-91eb-70cc84ba3226 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:34.708570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da8a2bf-f91e-459f-91eb-70cc84ba3226 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:34.740479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0dc27b-f415-41ab-94fc-dd083b0b9011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:34.741334 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0dc27b-f415-41ab-94fc-dd083b0b9011 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:34.741620 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0dc27b-f415-41ab-94fc-dd083b0b9011 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:34.741895 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0dc27b-f415-41ab-94fc-dd083b0b9011 None admin] PATH_INFO: `/v3/users/7031e51b30dd45d6bf84bfa4dc04ab86` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:34.743059 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0dc27b-f415-41ab-94fc-dd083b0b9011 None admin] RBAC: Authorizing `identity:get_user(user_id=7031e51b30dd45d6bf84bfa4dc04ab86)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:34.744523 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0dc27b-f415-41ab-94fc-dd083b0b9011 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:34.746114 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 606/1190] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:04:34 2022] GET /identity/v3/users/7031e51b30dd45d6bf84bfa4dc04ab86 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:04:34.752188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b04aa28-2d0d-4b5c-b000-1644093eae74 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:34.833258 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:34.848916 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b04aa28-2d0d-4b5c-b000-1644093eae74 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:34.881915 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83bb7ed-7ee1-4edf-9208-6e3e91ee12f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:34.882768 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bb7ed-7ee1-4edf-9208-6e3e91ee12f4 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:34.883056 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bb7ed-7ee1-4edf-9208-6e3e91ee12f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:34.883359 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bb7ed-7ee1-4edf-9208-6e3e91ee12f4 None admin] PATH_INFO: `/v3/projects/688971680e16429eb655e00d8fbe95fa` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:34.884528 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83bb7ed-7ee1-4edf-9208-6e3e91ee12f4 None admin] RBAC: Authorizing `identity:get_project(project_id=688971680e16429eb655e00d8fbe95fa)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:34.885945 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83bb7ed-7ee1-4edf-9208-6e3e91ee12f4 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:34.887668 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 585/1191] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:04:34 2022] GET /identity/v3/projects/688971680e16429eb655e00d8fbe95fa => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:04:34.893369 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0dc27b-f415-41ab-94fc-dd083b0b9011 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:34.897061 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:34.990537 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0dc27b-f415-41ab-94fc-dd083b0b9011 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:35.022977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ca312-22a6-43b2-9224-45cbf9597203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:35.023879 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ca312-22a6-43b2-9224-45cbf9597203 None admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:35.024173 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ca312-22a6-43b2-9224-45cbf9597203 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:35.024452 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ca312-22a6-43b2-9224-45cbf9597203 None admin] PATH_INFO: `/v3/projects/688971680e16429eb655e00d8fbe95fa/users/7031e51b30dd45d6bf84bfa4dc04ab86/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:35.026634 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75ca312-22a6-43b2-9224-45cbf9597203 None admin] RBAC: Authorizing `identity:create_grant(project_id=688971680e16429eb655e00d8fbe95fa, user_id=7031e51b30dd45d6bf84bfa4dc04ab86, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:35.028081 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75ca312-22a6-43b2-9224-45cbf9597203 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:35.040105 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 607/1192] 10.222.0.41 () {58 vars in 1439 bytes} [Fri Jun 3 16:04:34 2022] PUT /identity/v3/projects/688971680e16429eb655e00d8fbe95fa/users/7031e51b30dd45d6bf84bfa4dc04ab86/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:04:37.227472 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8f140-ee96-4736-a03c-3306f96c4d60 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:37.227989 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8f140-ee96-4736-a03c-3306f96c4d60 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:37.228116 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8f140-ee96-4736-a03c-3306f96c4d60 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:37.229387 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 586/1193] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:04:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:04:37.235136 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3ae88-334a-445f-9fd4-0e9208c2d932 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:37.235557 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3ae88-334a-445f-9fd4-0e9208c2d932 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:37.235729 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3ae88-334a-445f-9fd4-0e9208c2d932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:37.253222 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-c3a3ae88-334a-445f-9fd4-0e9208c2d932 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:04:37.254390 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-c3a3ae88-334a-445f-9fd4-0e9208c2d932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:04:37.310620 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:37.320088 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 608/1194] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:04:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:04:37.344173 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8f140-ee96-4736-a03c-3306f96c4d60 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:37.442052 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8f140-ee96-4736-a03c-3306f96c4d60 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:37.473778 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5caa94b-776b-4a75-b305-046644e928ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:37.474541 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5caa94b-776b-4a75-b305-046644e928ea None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:37.474821 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5caa94b-776b-4a75-b305-046644e928ea None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:37.475095 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5caa94b-776b-4a75-b305-046644e928ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:37.485980 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5caa94b-776b-4a75-b305-046644e928ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:04:37.493322 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-a5caa94b-776b-4a75-b305-046644e928ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:04:37.493322 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:04:37.493322 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:04:37.493322 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:04:37.493322 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:04:37.493322 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:04:37.493322 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:04:37.493322 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:04:37.493322 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:04:37.493322 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:04:37.493322 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:04:37.493322 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:04:37.493322 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:04:37.493322 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:04:37.493322 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:04:37.493322 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:04:37.494449 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:04:37.494449 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:04:37.494449 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:04:37.494449 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:04:37.494449 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:04:37.494449 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:04:37.494449 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:04:37.494449 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:04:37.494449 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:04:37.494449 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:04:37.494449 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:04:37.494449 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:04:37.494449 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:04:37.494449 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:04:37.494449 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:04:37.494449 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:04:37.495520 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:04:37.495520 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:04:37.495520 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:04:37.495520 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:04:37.495520 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:04:37.495520 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:04:37.495520 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:04:37.495520 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:04:37.495520 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:04:37.495520 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:04:37.495520 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:04:37.495520 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:04:37.495520 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 587/1195] 10.222.0.41 () {56 vars in 1096 bytes} [Fri Jun 3 16:04:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:04:37.500857 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3ae88-334a-445f-9fd4-0e9208c2d932 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:37.594674 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3ae88-334a-445f-9fd4-0e9208c2d932 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:37.626662 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107cb939-3241-440b-93a5-63a86105b7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:37.627343 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107cb939-3241-440b-93a5-63a86105b7d9 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:37.627622 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107cb939-3241-440b-93a5-63a86105b7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:37.627892 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107cb939-3241-440b-93a5-63a86105b7d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:37.628622 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107cb939-3241-440b-93a5-63a86105b7d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:04:37.628923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107cb939-3241-440b-93a5-63a86105b7d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:37.630335 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107cb939-3241-440b-93a5-63a86105b7d9 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:37.638775 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 609/1196] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:04:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:04:37.644316 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5caa94b-776b-4a75-b305-046644e928ea None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:37.659926 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:37.741616 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5caa94b-776b-4a75-b305-046644e928ea None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:37.773974 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2879e2f4-b9d2-4610-9f8c-273ff8f0977d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:37.774853 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879e2f4-b9d2-4610-9f8c-273ff8f0977d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:37.775140 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879e2f4-b9d2-4610-9f8c-273ff8f0977d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:37.775450 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879e2f4-b9d2-4610-9f8c-273ff8f0977d None admin] PATH_INFO: `/v3/users/7031e51b30dd45d6bf84bfa4dc04ab86` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:37.776888 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2879e2f4-b9d2-4610-9f8c-273ff8f0977d None admin] RBAC: Authorizing `identity:get_user(user_id=7031e51b30dd45d6bf84bfa4dc04ab86)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:37.778333 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2879e2f4-b9d2-4610-9f8c-273ff8f0977d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:37.780083 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 588/1197] 10.222.0.41 () {56 vars in 1177 bytes} [Fri Jun 3 16:04:37 2022] GET /identity/v3/users/7031e51b30dd45d6bf84bfa4dc04ab86 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:04:37.785977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107cb939-3241-440b-93a5-63a86105b7d9 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:37.840720 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:37.883210 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107cb939-3241-440b-93a5-63a86105b7d9 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:37.914521 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecc3d4c-f74c-4faa-aefa-9f1aec34984f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:37.915147 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc3d4c-f74c-4faa-aefa-9f1aec34984f None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:37.915457 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc3d4c-f74c-4faa-aefa-9f1aec34984f None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:37.915734 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc3d4c-f74c-4faa-aefa-9f1aec34984f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:37.979264 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecc3d4c-f74c-4faa-aefa-9f1aec34984f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:37.980715 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecc3d4c-f74c-4faa-aefa-9f1aec34984f None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:38.033584 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:38.076963 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 610/1198] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:04:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 16:04:38.082906 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2879e2f4-b9d2-4610-9f8c-273ff8f0977d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:38.159710 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:38.180046 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2879e2f4-b9d2-4610-9f8c-273ff8f0977d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:38.218548 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5da003-3a17-4630-b489-a1ed177ca937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:38.219418 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5da003-3a17-4630-b489-a1ed177ca937 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:38.219702 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5da003-3a17-4630-b489-a1ed177ca937 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:38.219973 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5da003-3a17-4630-b489-a1ed177ca937 None admin] PATH_INFO: `/v3/projects/688971680e16429eb655e00d8fbe95fa` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:38.221127 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5da003-3a17-4630-b489-a1ed177ca937 None admin] RBAC: Authorizing `identity:get_project(project_id=688971680e16429eb655e00d8fbe95fa)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:38.222536 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5da003-3a17-4630-b489-a1ed177ca937 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:38.224283 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 589/1199] 10.222.0.41 () {56 vars in 1186 bytes} [Fri Jun 3 16:04:38 2022] GET /identity/v3/projects/688971680e16429eb655e00d8fbe95fa => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:04:38.230316 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecc3d4c-f74c-4faa-aefa-9f1aec34984f None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:04:38.325678 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecc3d4c-f74c-4faa-aefa-9f1aec34984f None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:04:38.358794 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63926900-9f69-4952-8257-39957910d081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:04:38.359475 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63926900-9f69-4952-8257-39957910d081 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:04:38.359763 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63926900-9f69-4952-8257-39957910d081 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:04:38.360043 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63926900-9f69-4952-8257-39957910d081 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:04:38.360865 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63926900-9f69-4952-8257-39957910d081 None admin] RBAC: Adding query filter params (role.id=75936a608b8149099ff54e317f3a6740, scope.project.id=688971680e16429eb655e00d8fbe95fa, user.id=7031e51b30dd45d6bf84bfa4dc04ab86) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:04:38.361178 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63926900-9f69-4952-8257-39957910d081 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:04:38.362584 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63926900-9f69-4952-8257-39957910d081 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:04:38.368279 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 611/1200] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:04:38 2022] GET /identity/v3/role_assignments?user.id=7031e51b30dd45d6bf84bfa4dc04ab86&scope.project.id=688971680e16429eb655e00d8fbe95fa&role.id=75936a608b8149099ff54e317f3a6740 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:04:48.213982 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=26 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:04:48.365764 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=90 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:39.582913 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91518f53-6f5d-4786-9d3a-658cfd4c0884 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:39.583492 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91518f53-6f5d-4786-9d3a-658cfd4c0884 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:39.583492 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91518f53-6f5d-4786-9d3a-658cfd4c0884 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:39.584782 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 590/1201] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:05:39.590904 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811d39d-84da-4d5d-a188-e630f4928e47 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:39.591977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811d39d-84da-4d5d-a188-e630f4928e47 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:39.592080 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811d39d-84da-4d5d-a188-e630f4928e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:39.612272 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-0811d39d-84da-4d5d-a188-e630f4928e47 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:05:39.613809 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-0811d39d-84da-4d5d-a188-e630f4928e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:05:39.708400 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 612/1202] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:05:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:05:39.736409 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91518f53-6f5d-4786-9d3a-658cfd4c0884 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:39.870108 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91518f53-6f5d-4786-9d3a-658cfd4c0884 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:39.917430 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34735dde-2b62-4efa-afa1-7a954209090a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:39.918438 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34735dde-2b62-4efa-afa1-7a954209090a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:39.918754 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34735dde-2b62-4efa-afa1-7a954209090a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:39.919040 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34735dde-2b62-4efa-afa1-7a954209090a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:39.930761 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34735dde-2b62-4efa-afa1-7a954209090a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:39.932268 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34735dde-2b62-4efa-afa1-7a954209090a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:39.954135 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-34735dde-2b62-4efa-afa1-7a954209090a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:05:39.954135 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:05:39.954135 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:05:39.954135 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:05:39.954135 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:05:39.954135 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:05:39.954135 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:05:39.954135 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:05:39.954135 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:05:39.954135 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:05:39.954135 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:05:39.954135 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:05:39.954135 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:05:39.954135 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:05:39.954135 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:05:39.956126 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:05:39.956126 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:05:39.956126 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:05:39.956126 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:05:39.956126 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:05:39.956126 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:05:39.956126 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:05:39.956126 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:05:39.956126 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:05:39.956126 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:05:39.956126 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:05:39.956126 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:05:39.956126 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:05:39.956126 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:05:39.956126 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:05:39.956126 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:05:39.956126 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:05:39.957970 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:05:39.957970 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:05:39.957970 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:05:39.957970 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:05:39.957970 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 591/1203] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:05:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:05:39.967940 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811d39d-84da-4d5d-a188-e630f4928e47 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:40.107235 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811d39d-84da-4d5d-a188-e630f4928e47 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:40.143110 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:40.154683 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09167fa1-5f5a-4944-8b3d-0571f714f7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:40.155165 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09167fa1-5f5a-4944-8b3d-0571f714f7d6 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:40.155482 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09167fa1-5f5a-4944-8b3d-0571f714f7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:40.155764 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09167fa1-5f5a-4944-8b3d-0571f714f7d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:40.156494 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09167fa1-5f5a-4944-8b3d-0571f714f7d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:05:40.156807 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09167fa1-5f5a-4944-8b3d-0571f714f7d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:40.158187 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09167fa1-5f5a-4944-8b3d-0571f714f7d6 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:40.176822 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 613/1204] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:05:40.183035 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34735dde-2b62-4efa-afa1-7a954209090a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:40.251326 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:40.305628 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34735dde-2b62-4efa-afa1-7a954209090a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:40.346122 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb862fe3-a724-4d90-bacf-3db6e8e7c11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:40.347174 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb862fe3-a724-4d90-bacf-3db6e8e7c11f None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:40.347709 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb862fe3-a724-4d90-bacf-3db6e8e7c11f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:40.348206 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb862fe3-a724-4d90-bacf-3db6e8e7c11f None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:40.349142 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb862fe3-a724-4d90-bacf-3db6e8e7c11f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:40.350788 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb862fe3-a724-4d90-bacf-3db6e8e7c11f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:40.374830 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 592/1205] 10.222.0.41 () {60 vars in 1132 bytes} [Fri Jun 3 16:05:40 2022] POST /identity/v3/users => generated 251 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:05:42.956848 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f25d4a-fb64-48bc-9ec8-a1dd8526fb40 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:42.957534 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f25d4a-fb64-48bc-9ec8-a1dd8526fb40 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:42.957632 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f25d4a-fb64-48bc-9ec8-a1dd8526fb40 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:42.959115 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 614/1206] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:05:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 16:05:42.964419 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114e564-6083-4f94-8a74-0fc9eb267a66 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:42.966598 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114e564-6083-4f94-8a74-0fc9eb267a66 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:42.966598 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114e564-6083-4f94-8a74-0fc9eb267a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:42.987539 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-5114e564-6083-4f94-8a74-0fc9eb267a66 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:05:42.988797 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5114e564-6083-4f94-8a74-0fc9eb267a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:05:43.075085 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 593/1207] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:05:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:05:43.102438 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f25d4a-fb64-48bc-9ec8-a1dd8526fb40 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:43.221685 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f25d4a-fb64-48bc-9ec8-a1dd8526fb40 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:43.261620 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6360a3f-61ac-4981-9f4f-a9958a5cd22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:43.262571 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6360a3f-61ac-4981-9f4f-a9958a5cd22f None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:43.263590 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6360a3f-61ac-4981-9f4f-a9958a5cd22f None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:43.263891 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6360a3f-61ac-4981-9f4f-a9958a5cd22f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:43.276559 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6360a3f-61ac-4981-9f4f-a9958a5cd22f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:05:43.285811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-b6360a3f-61ac-4981-9f4f-a9958a5cd22f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:05:43.285811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:05:43.285811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:05:43.285811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:05:43.285811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:05:43.285811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:05:43.285811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:05:43.285811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:05:43.285811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:05:43.285811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:05:43.285811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:05:43.285811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:05:43.285811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:05:43.285811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:05:43.285811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:05:43.285811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:05:43.286829 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:05:43.286829 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:05:43.286829 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:05:43.286829 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:05:43.286829 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:05:43.286829 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:05:43.286829 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:05:43.286829 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:05:43.286829 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:05:43.286829 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:05:43.286829 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:05:43.286829 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:05:43.286829 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:05:43.286829 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:05:43.286829 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:05:43.286829 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:05:43.288421 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:05:43.288421 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:05:43.288421 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:05:43.288421 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:05:43.288421 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:05:43.288421 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:05:43.288421 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:05:43.288421 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:05:43.288421 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:05:43.288421 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:05:43.288421 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:05:43.288421 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:05:43.288421 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 615/1208] 10.222.0.41 () {56 vars in 1102 bytes} [Fri Jun 3 16:05:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:05:43.293323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114e564-6083-4f94-8a74-0fc9eb267a66 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:43.343380 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:43.413351 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114e564-6083-4f94-8a74-0fc9eb267a66 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:43.452609 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68851b-615a-46b1-8aae-f580a3a7f1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:43.453286 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68851b-615a-46b1-8aae-f580a3a7f1a8 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:43.453571 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68851b-615a-46b1-8aae-f580a3a7f1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:43.453848 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68851b-615a-46b1-8aae-f580a3a7f1a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:43.454598 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf68851b-615a-46b1-8aae-f580a3a7f1a8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:05:43.454925 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf68851b-615a-46b1-8aae-f580a3a7f1a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:43.456346 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf68851b-615a-46b1-8aae-f580a3a7f1a8 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:43.465108 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 594/1209] 10.222.0.41 () {56 vars in 1103 bytes} [Fri Jun 3 16:05:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:05:43.470985 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6360a3f-61ac-4981-9f4f-a9958a5cd22f None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:43.533119 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:43.589549 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6360a3f-61ac-4981-9f4f-a9958a5cd22f None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:43.630202 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be56b86-c4c7-46bc-aa71-bdcbacf66c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:43.631012 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be56b86-c4c7-46bc-aa71-bdcbacf66c0e None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:43.631316 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be56b86-c4c7-46bc-aa71-bdcbacf66c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:43.631598 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be56b86-c4c7-46bc-aa71-bdcbacf66c0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:43.642969 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be56b86-c4c7-46bc-aa71-bdcbacf66c0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:43.644494 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be56b86-c4c7-46bc-aa71-bdcbacf66c0e None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:43.667738 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-5be56b86-c4c7-46bc-aa71-bdcbacf66c0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:05:43.667738 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:05:43.667738 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:05:43.667738 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:05:43.667738 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:05:43.667738 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:05:43.667738 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:05:43.667738 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:05:43.667738 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:05:43.667738 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:05:43.667738 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:05:43.667738 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:05:43.667738 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:05:43.667738 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:05:43.667738 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:05:43.668784 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:05:43.668784 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:05:43.668784 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:05:43.668784 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:05:43.668784 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:05:43.668784 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:05:43.668784 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:05:43.668784 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:05:43.668784 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:05:43.668784 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:05:43.668784 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:05:43.668784 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:05:43.668784 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:05:43.668784 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:05:43.668784 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:05:43.668784 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:05:43.668784 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:05:43.670297 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:05:43.670297 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:05:43.670297 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:05:43.670297 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:05:43.670297 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 616/1210] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:05:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:05:43.675901 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68851b-615a-46b1-8aae-f580a3a7f1a8 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:43.782840 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:43.788934 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68851b-615a-46b1-8aae-f580a3a7f1a8 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:43.823933 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b0354f-19e5-4085-8833-000744fa1c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:43.825202 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0354f-19e5-4085-8833-000744fa1c3e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:43.825202 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0354f-19e5-4085-8833-000744fa1c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:43.825490 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0354f-19e5-4085-8833-000744fa1c3e None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:43.826057 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b0354f-19e5-4085-8833-000744fa1c3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:05:43.826371 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b0354f-19e5-4085-8833-000744fa1c3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:43.827773 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b0354f-19e5-4085-8833-000744fa1c3e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:43.840076 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 595/1211] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:05:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:05:43.845896 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be56b86-c4c7-46bc-aa71-bdcbacf66c0e None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:43.954229 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be56b86-c4c7-46bc-aa71-bdcbacf66c0e None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:43.955914 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:43.998668 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f9d7d-368d-49ff-a7e4-e3948f6a4277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:44.000055 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f9d7d-368d-49ff-a7e4-e3948f6a4277 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:44.000400 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f9d7d-368d-49ff-a7e4-e3948f6a4277 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:44.000846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f9d7d-368d-49ff-a7e4-e3948f6a4277 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:44.011190 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f9d7d-368d-49ff-a7e4-e3948f6a4277 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:44.013040 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f9d7d-368d-49ff-a7e4-e3948f6a4277 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:44.034173 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-986f9d7d-368d-49ff-a7e4-e3948f6a4277 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 03 16:05:44.034173 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:05:44.034173 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:05:44.034173 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:05:44.034173 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:05:44.034173 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:05:44.034173 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:05:44.034173 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:05:44.034173 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:05:44.034173 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:05:44.034173 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:05:44.034173 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:05:44.034173 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:05:44.034173 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:05:44.034173 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:05:44.034173 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:05:44.035260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:05:44.035260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:05:44.035260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:05:44.035260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:05:44.035260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:05:44.035260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:05:44.035260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:05:44.035260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:05:44.035260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:05:44.035260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:05:44.035260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:05:44.035260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:05:44.035260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:05:44.035260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:05:44.035260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:05:44.035260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:05:44.035260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:05:44.035260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:05:44.037336 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:05:44.037336 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:05:44.037336 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:05:44.037336 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:05:44.037336 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:05:44.037336 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:05:44.037336 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:05:44.037336 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:05:44.037336 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 03 16:05:44.037336 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:05:44.037336 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 617/1212] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:05:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:05:44.043427 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b0354f-19e5-4085-8833-000744fa1c3e None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:44.157001 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b0354f-19e5-4085-8833-000744fa1c3e None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:44.204380 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7039a9-90c2-4037-9256-971d3ac73c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:44.205059 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7039a9-90c2-4037-9256-971d3ac73c2c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:44.205344 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7039a9-90c2-4037-9256-971d3ac73c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:44.205591 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7039a9-90c2-4037-9256-971d3ac73c2c None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:44.206423 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7039a9-90c2-4037-9256-971d3ac73c2c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:05:44.206829 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7039a9-90c2-4037-9256-971d3ac73c2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:44.208270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7039a9-90c2-4037-9256-971d3ac73c2c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:44.232751 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 596/1213] 10.222.0.41 () {56 vars in 1143 bytes} [Fri Jun 3 16:05:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:05:44.238729 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f9d7d-368d-49ff-a7e4-e3948f6a4277 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:44.347113 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f9d7d-368d-49ff-a7e4-e3948f6a4277 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:44.381485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee58181-e565-4b3e-ab7d-88bd071446b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:44.382141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee58181-e565-4b3e-ab7d-88bd071446b8 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:44.382427 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee58181-e565-4b3e-ab7d-88bd071446b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:44.382704 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee58181-e565-4b3e-ab7d-88bd071446b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:44.416345 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:44.458788 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee58181-e565-4b3e-ab7d-88bd071446b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:44.460468 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee58181-e565-4b3e-ab7d-88bd071446b8 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:44.566604 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 618/1214] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:05:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 16:05:44.573352 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7039a9-90c2-4037-9256-971d3ac73c2c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:44.594341 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:44.684549 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7039a9-90c2-4037-9256-971d3ac73c2c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:44.723440 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db98c4-85e9-4054-ac5c-b35ee8357274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:44.724640 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db98c4-85e9-4054-ac5c-b35ee8357274 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:44.725193 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db98c4-85e9-4054-ac5c-b35ee8357274 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:44.725686 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db98c4-85e9-4054-ac5c-b35ee8357274 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:44.737823 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53db98c4-85e9-4054-ac5c-b35ee8357274 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:44.739534 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53db98c4-85e9-4054-ac5c-b35ee8357274 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:44.758178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-53db98c4-85e9-4054-ac5c-b35ee8357274 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:05:44.758178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:05:44.758178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:05:44.758178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:05:44.758178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:05:44.758178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:05:44.758178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:05:44.758178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:05:44.758178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:05:44.758178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:05:44.758178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:05:44.758178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:05:44.758178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:05:44.758178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:05:44.758178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:05:44.759177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:05:44.759177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:05:44.759177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:05:44.759177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:05:44.759177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:05:44.759177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:05:44.759177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:05:44.759177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:05:44.759177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:05:44.759177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:05:44.759177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:05:44.759177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:05:44.759177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:05:44.759177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:05:44.759177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:05:44.759177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:05:44.759177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:05:44.760111 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:05:44.760111 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:05:44.760111 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:05:44.760111 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:05:44.760111 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 597/1215] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:05:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:05:44.765186 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee58181-e565-4b3e-ab7d-88bd071446b8 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:44.834294 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:44.881314 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee58181-e565-4b3e-ab7d-88bd071446b8 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:44.919816 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f6cb2-071e-41ab-81b8-acb66e5b091f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:44.920487 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f6cb2-071e-41ab-81b8-acb66e5b091f None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:44.920772 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f6cb2-071e-41ab-81b8-acb66e5b091f None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:44.921048 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f6cb2-071e-41ab-81b8-acb66e5b091f None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:44.921762 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f6cb2-071e-41ab-81b8-acb66e5b091f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:05:44.922075 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f6cb2-071e-41ab-81b8-acb66e5b091f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:44.923488 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f6cb2-071e-41ab-81b8-acb66e5b091f None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:44.936015 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 619/1216] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:05:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:05:44.941459 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db98c4-85e9-4054-ac5c-b35ee8357274 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:45.022401 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:45.063452 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db98c4-85e9-4054-ac5c-b35ee8357274 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:45.110376 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841fa3f9-466e-471a-b45b-dd01fc9bda38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:45.111412 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841fa3f9-466e-471a-b45b-dd01fc9bda38 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:45.111997 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841fa3f9-466e-471a-b45b-dd01fc9bda38 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:45.112369 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841fa3f9-466e-471a-b45b-dd01fc9bda38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:45.121627 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841fa3f9-466e-471a-b45b-dd01fc9bda38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:45.124236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841fa3f9-466e-471a-b45b-dd01fc9bda38 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:45.141667 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-841fa3f9-466e-471a-b45b-dd01fc9bda38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:05:45.141667 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:05:45.141667 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:05:45.141667 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:05:45.141667 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:05:45.141667 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:05:45.141667 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:05:45.141667 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:05:45.141667 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:05:45.141667 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:05:45.141667 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:05:45.141667 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:05:45.141667 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:05:45.141667 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:05:45.141667 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:05:45.142885 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:05:45.142885 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:05:45.142885 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:05:45.142885 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:05:45.142885 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:05:45.142885 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:05:45.142885 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:05:45.142885 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:05:45.142885 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:05:45.142885 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:05:45.142885 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:05:45.142885 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:05:45.142885 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:05:45.142885 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:05:45.142885 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:05:45.142885 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:05:45.142885 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:05:45.144007 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:05:45.144007 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:05:45.144007 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:05:45.144007 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:05:45.144007 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:05:45.144007 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:05:45.144007 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:05:45.144007 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:05:45.145633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 598/1217] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:05:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:05:45.152691 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f6cb2-071e-41ab-81b8-acb66e5b091f None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:45.277487 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f6cb2-071e-41ab-81b8-acb66e5b091f None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:45.306643 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:45.318813 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8502c-5355-4da4-acd8-7494689ba050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:45.318813 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8502c-5355-4da4-acd8-7494689ba050 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:45.318813 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8502c-5355-4da4-acd8-7494689ba050 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:45.319236 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8502c-5355-4da4-acd8-7494689ba050 None admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:45.319723 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8502c-5355-4da4-acd8-7494689ba050 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:05:45.320034 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8502c-5355-4da4-acd8-7494689ba050 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:45.321456 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8502c-5355-4da4-acd8-7494689ba050 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:45.335270 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 620/1218] 10.222.0.41 () {56 vars in 1148 bytes} [Fri Jun 3 16:05:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:05:45.341402 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841fa3f9-466e-471a-b45b-dd01fc9bda38 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:45.466715 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841fa3f9-466e-471a-b45b-dd01fc9bda38 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:45.498360 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:45.504914 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a2dde-e4c9-4976-ae1d-772df0be7aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:45.505549 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a2dde-e4c9-4976-ae1d-772df0be7aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:45.505799 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a2dde-e4c9-4976-ae1d-772df0be7aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:45.506070 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a2dde-e4c9-4976-ae1d-772df0be7aa2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:45.506902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3a2dde-e4c9-4976-ae1d-772df0be7aa2 None admin] RBAC: Adding query filter params (role.id=fe695aedbe8249a28c992a19264bc34b, scope.project.id=c274254e510a468e9acbecc6298301cc, user.id=be901786f8934d559e0bac5d622f97cb) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:05:45.507216 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3a2dde-e4c9-4976-ae1d-772df0be7aa2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:45.508685 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3a2dde-e4c9-4976-ae1d-772df0be7aa2 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:45.523337 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 599/1219] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:05:45 2022] GET /identity/v3/role_assignments?user.id=be901786f8934d559e0bac5d622f97cb&scope.project.id=c274254e510a468e9acbecc6298301cc&role.id=fe695aedbe8249a28c992a19264bc34b => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:05:47.853401 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0828f-58b1-40e4-bf24-80e668ba7ad3 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:47.853964 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0828f-58b1-40e4-bf24-80e668ba7ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:47.853964 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0828f-58b1-40e4-bf24-80e668ba7ad3 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:47.855529 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 621/1220] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:05:47.861855 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f73ba-bdf6-4e09-9fc4-42ddc5669ac5 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:47.862362 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f73ba-bdf6-4e09-9fc4-42ddc5669ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:47.862466 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f73ba-bdf6-4e09-9fc4-42ddc5669ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:47.881507 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b28f73ba-bdf6-4e09-9fc4-42ddc5669ac5 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:05:47.883211 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b28f73ba-bdf6-4e09-9fc4-42ddc5669ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:05:47.973822 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 600/1221] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:05:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:05:47.999492 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc0828f-58b1-40e4-bf24-80e668ba7ad3 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:48.115801 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc0828f-58b1-40e4-bf24-80e668ba7ad3 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:48.154782 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfcada-473a-4eae-abcb-e15e42036157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:48.155616 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfcada-473a-4eae-abcb-e15e42036157 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:48.155894 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfcada-473a-4eae-abcb-e15e42036157 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:48.156162 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfcada-473a-4eae-abcb-e15e42036157 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:48.166768 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bfcada-473a-4eae-abcb-e15e42036157 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:05:48.175623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-a1bfcada-473a-4eae-abcb-e15e42036157 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:05:48.175623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:05:48.175623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:05:48.175623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:05:48.175623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:05:48.175623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:05:48.175623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:05:48.175623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:05:48.175623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:05:48.175623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:05:48.175623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:05:48.175623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:05:48.175623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:05:48.175623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:05:48.175623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:05:48.175623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:05:48.176839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:05:48.176839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:05:48.176839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:05:48.176839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:05:48.176839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:05:48.176839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:05:48.176839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:05:48.176839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:05:48.176839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:05:48.176839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:05:48.176839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:05:48.176839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:05:48.176839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:05:48.176839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:05:48.176839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:05:48.176839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:05:48.177846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:05:48.177846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:05:48.177846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:05:48.177846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:05:48.177846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:05:48.177846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:05:48.177846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:05:48.177846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:05:48.177846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:05:48.177846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:05:48.177846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:05:48.177846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:05:48.177846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 622/1222] 10.222.0.41 () {56 vars in 1102 bytes} [Fri Jun 3 16:05:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:05:48.183657 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28f73ba-bdf6-4e09-9fc4-42ddc5669ac5 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:48.298448 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28f73ba-bdf6-4e09-9fc4-42ddc5669ac5 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:48.326950 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:48.339111 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b39528-8233-4553-81b4-de714087984e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:48.340152 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b39528-8233-4553-81b4-de714087984e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:48.341135 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b39528-8233-4553-81b4-de714087984e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:48.341414 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b39528-8233-4553-81b4-de714087984e None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:48.342511 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b39528-8233-4553-81b4-de714087984e None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:05:48.342830 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b39528-8233-4553-81b4-de714087984e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:48.344724 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b39528-8233-4553-81b4-de714087984e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:48.353867 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 601/1223] 10.222.0.41 () {56 vars in 1103 bytes} [Fri Jun 3 16:05:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:05:48.359976 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfcada-473a-4eae-abcb-e15e42036157 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:48.413271 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:48.487037 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfcada-473a-4eae-abcb-e15e42036157 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:48.527194 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0bda0-49b1-4e8a-aa33-8012f4a2e075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:48.528880 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0bda0-49b1-4e8a-aa33-8012f4a2e075 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:48.529351 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0bda0-49b1-4e8a-aa33-8012f4a2e075 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:48.529848 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0bda0-49b1-4e8a-aa33-8012f4a2e075 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:48.541591 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce0bda0-49b1-4e8a-aa33-8012f4a2e075 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:48.543026 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce0bda0-49b1-4e8a-aa33-8012f4a2e075 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:48.566637 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-6ce0bda0-49b1-4e8a-aa33-8012f4a2e075 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:05:48.566637 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:05:48.566637 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:05:48.566637 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:05:48.566637 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:05:48.566637 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:05:48.566637 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:05:48.566637 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:05:48.566637 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:05:48.566637 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:05:48.566637 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:05:48.566637 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:05:48.566637 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:05:48.566637 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:05:48.566637 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:05:48.568011 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:05:48.568011 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:05:48.568011 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:05:48.568011 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:05:48.568011 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:05:48.568011 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:05:48.568011 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:05:48.568011 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:05:48.568011 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:05:48.568011 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:05:48.568011 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:05:48.568011 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:05:48.568011 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:05:48.568011 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:05:48.568011 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:05:48.568011 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:05:48.568011 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:05:48.569109 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:05:48.569109 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:05:48.569109 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:05:48.569109 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:05:48.569109 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 623/1224] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:05:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:05:48.575178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b39528-8233-4553-81b4-de714087984e None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:48.697074 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b39528-8233-4553-81b4-de714087984e None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:48.734906 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d0c563-8edf-45e2-9e95-db01f2a6154c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:48.735610 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0c563-8edf-45e2-9e95-db01f2a6154c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:48.735893 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0c563-8edf-45e2-9e95-db01f2a6154c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:48.736182 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0c563-8edf-45e2-9e95-db01f2a6154c None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:48.736991 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d0c563-8edf-45e2-9e95-db01f2a6154c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:05:48.737304 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d0c563-8edf-45e2-9e95-db01f2a6154c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:48.738731 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d0c563-8edf-45e2-9e95-db01f2a6154c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:48.752454 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 602/1225] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:05:48.758426 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0bda0-49b1-4e8a-aa33-8012f4a2e075 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:48.870273 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:48.877754 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0bda0-49b1-4e8a-aa33-8012f4a2e075 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:48.917781 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b692c-cc65-46ec-952b-d87b63dcace5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:48.918579 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b692c-cc65-46ec-952b-d87b63dcace5 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:48.918903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b692c-cc65-46ec-952b-d87b63dcace5 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:48.919176 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b692c-cc65-46ec-952b-d87b63dcace5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:48.926715 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090b692c-cc65-46ec-952b-d87b63dcace5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:48.928177 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090b692c-cc65-46ec-952b-d87b63dcace5 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:48.942227 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-090b692c-cc65-46ec-952b-d87b63dcace5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 03 16:05:48.942227 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:05:48.942227 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:05:48.942227 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:05:48.942227 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:05:48.942227 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:05:48.942227 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:05:48.942227 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:05:48.942227 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:05:48.942227 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:05:48.942227 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:05:48.942227 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:05:48.942227 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:05:48.942227 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:05:48.942227 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:05:48.942227 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:05:48.943371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:05:48.943371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:05:48.943371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:05:48.943371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:05:48.943371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:05:48.943371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:05:48.943371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:05:48.943371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:05:48.943371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:05:48.943371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:05:48.943371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:05:48.943371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:05:48.943371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:05:48.943371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:05:48.943371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:05:48.943371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:05:48.943371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:05:48.943371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:05:48.944917 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:05:48.944917 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:05:48.944917 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:05:48.944917 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:05:48.944917 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:05:48.944917 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:05:48.944917 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:05:48.944917 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:05:48.944917 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 03 16:05:48.944917 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:05:48.944917 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 624/1226] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:05:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:05:48.950465 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d0c563-8edf-45e2-9e95-db01f2a6154c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:49.001729 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:49.070332 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d0c563-8edf-45e2-9e95-db01f2a6154c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:49.108776 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173180ce-763a-46b4-905e-26ff7f9e58fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:49.109455 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173180ce-763a-46b4-905e-26ff7f9e58fc None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:49.109762 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173180ce-763a-46b4-905e-26ff7f9e58fc None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:49.110017 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173180ce-763a-46b4-905e-26ff7f9e58fc None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:49.110850 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173180ce-763a-46b4-905e-26ff7f9e58fc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:05:49.111159 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173180ce-763a-46b4-905e-26ff7f9e58fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:49.112634 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173180ce-763a-46b4-905e-26ff7f9e58fc None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:49.132802 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 603/1227] 10.222.0.41 () {56 vars in 1143 bytes} [Fri Jun 3 16:05:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:05:49.139094 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b692c-cc65-46ec-952b-d87b63dcace5 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:49.254478 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b692c-cc65-46ec-952b-d87b63dcace5 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:49.293211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b83be-adf1-47a4-95c3-2ee999320280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:49.294644 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b83be-adf1-47a4-95c3-2ee999320280 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:49.294929 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b83be-adf1-47a4-95c3-2ee999320280 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:49.295205 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b83be-adf1-47a4-95c3-2ee999320280 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:49.307897 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8b83be-adf1-47a4-95c3-2ee999320280 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:49.309470 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8b83be-adf1-47a4-95c3-2ee999320280 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:49.327954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-ef8b83be-adf1-47a4-95c3-2ee999320280 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:05:49.327954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:05:49.327954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:05:49.327954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:05:49.327954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:05:49.327954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:05:49.327954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:05:49.327954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:05:49.327954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:05:49.327954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:05:49.327954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:05:49.327954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:05:49.327954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:05:49.327954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:05:49.327954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:05:49.328909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:05:49.328909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:05:49.328909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:05:49.328909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:05:49.328909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:05:49.328909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:05:49.328909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:05:49.328909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:05:49.328909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:05:49.328909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:05:49.328909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:05:49.328909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:05:49.328909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:05:49.328909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:05:49.328909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:05:49.328909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:05:49.328909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:05:49.329817 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:05:49.329817 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:05:49.329817 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:05:49.329817 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:05:49.329817 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 625/1228] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:05:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:05:49.336299 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173180ce-763a-46b4-905e-26ff7f9e58fc None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:49.447158 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173180ce-763a-46b4-905e-26ff7f9e58fc None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:49.449528 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:49.482405 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bfe217-153d-47f2-acd0-810ad76b5898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:49.483065 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bfe217-153d-47f2-acd0-810ad76b5898 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:49.483383 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bfe217-153d-47f2-acd0-810ad76b5898 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:49.483664 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bfe217-153d-47f2-acd0-810ad76b5898 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:49.484380 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bfe217-153d-47f2-acd0-810ad76b5898 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:05:49.484689 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bfe217-153d-47f2-acd0-810ad76b5898 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:49.486053 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bfe217-153d-47f2-acd0-810ad76b5898 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:49.497651 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 604/1229] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:05:49.504199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b83be-adf1-47a4-95c3-2ee999320280 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:49.513852 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:49.618492 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b83be-adf1-47a4-95c3-2ee999320280 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:49.658043 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9a6be-bd87-4f93-9921-d96cc910a5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:49.658978 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9a6be-bd87-4f93-9921-d96cc910a5f1 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:49.659271 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9a6be-bd87-4f93-9921-d96cc910a5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:49.659603 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9a6be-bd87-4f93-9921-d96cc910a5f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:49.667395 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec9a6be-bd87-4f93-9921-d96cc910a5f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:49.669106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec9a6be-bd87-4f93-9921-d96cc910a5f1 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:49.684395 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-1ec9a6be-bd87-4f93-9921-d96cc910a5f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:05:49.684395 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:05:49.684395 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:05:49.684395 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:05:49.684395 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:05:49.684395 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:05:49.684395 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:05:49.684395 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:05:49.684395 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:05:49.684395 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:05:49.684395 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:05:49.684395 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:05:49.684395 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:05:49.684395 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:05:49.684395 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:05:49.685573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:05:49.685573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:05:49.685573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:05:49.685573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:05:49.685573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:05:49.685573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:05:49.685573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:05:49.685573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:05:49.685573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:05:49.685573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:05:49.685573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:05:49.685573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:05:49.685573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:05:49.685573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:05:49.685573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:05:49.685573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:05:49.685573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:05:49.686788 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:05:49.686788 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:05:49.686788 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:05:49.686788 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:05:49.686788 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:05:49.686788 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:05:49.686788 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:05:49.686788 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:05:49.686788 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 626/1230] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:05:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:05:49.691968 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bfe217-153d-47f2-acd0-810ad76b5898 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:49.800414 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bfe217-153d-47f2-acd0-810ad76b5898 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:49.837907 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad784f8-9640-41a3-b000-2022539440ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:49.838578 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad784f8-9640-41a3-b000-2022539440ed None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:49.838865 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad784f8-9640-41a3-b000-2022539440ed None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:49.839144 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad784f8-9640-41a3-b000-2022539440ed None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:49.840074 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad784f8-9640-41a3-b000-2022539440ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:05:49.840376 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad784f8-9640-41a3-b000-2022539440ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:49.842667 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad784f8-9640-41a3-b000-2022539440ed None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:49.855216 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 605/1231] 10.222.0.41 () {56 vars in 1148 bytes} [Fri Jun 3 16:05:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:05:49.861811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9a6be-bd87-4f93-9921-d96cc910a5f1 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:49.938475 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:49.974291 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9a6be-bd87-4f93-9921-d96cc910a5f1 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:50.013999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ebf5c-103b-4407-825b-ab8125f98aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:50.015120 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ebf5c-103b-4407-825b-ab8125f98aae None admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:50.015538 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ebf5c-103b-4407-825b-ab8125f98aae None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:50.015990 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ebf5c-103b-4407-825b-ab8125f98aae None admin] PATH_INFO: `/v3/projects/c274254e510a468e9acbecc6298301cc/users/be901786f8934d559e0bac5d622f97cb/roles/fe695aedbe8249a28c992a19264bc34b` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:50.035939 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04ebf5c-103b-4407-825b-ab8125f98aae None admin] RBAC: Authorizing `identity:create_grant(project_id=c274254e510a468e9acbecc6298301cc, user_id=be901786f8934d559e0bac5d622f97cb, role_id=fe695aedbe8249a28c992a19264bc34b)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:50.038686 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04ebf5c-103b-4407-825b-ab8125f98aae None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:50.054845 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 627/1232] 10.222.0.41 () {58 vars in 1439 bytes} [Fri Jun 3 16:05:49 2022] PUT /identity/v3/projects/c274254e510a468e9acbecc6298301cc/users/be901786f8934d559e0bac5d622f97cb/roles/fe695aedbe8249a28c992a19264bc34b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:05:52.536997 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ea4b8-f302-4a29-91c5-1aa2ceae351b None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:52.537547 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ea4b8-f302-4a29-91c5-1aa2ceae351b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:52.537547 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ea4b8-f302-4a29-91c5-1aa2ceae351b None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:52.538810 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 606/1233] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:05:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:05:52.545309 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a5efa-5395-4831-b839-c3f8633ded53 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:52.545764 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a5efa-5395-4831-b839-c3f8633ded53 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:52.545864 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a5efa-5395-4831-b839-c3f8633ded53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:52.564434 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-3e8a5efa-5395-4831-b839-c3f8633ded53 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:05:52.565655 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-3e8a5efa-5395-4831-b839-c3f8633ded53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:05:52.636516 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 628/1234] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:05:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:05:52.663225 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ea4b8-f302-4a29-91c5-1aa2ceae351b None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:52.689351 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:52.774813 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ea4b8-f302-4a29-91c5-1aa2ceae351b None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:52.813319 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae34ed-9d65-4928-8d3f-79984fc54670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:52.814042 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae34ed-9d65-4928-8d3f-79984fc54670 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:52.814328 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae34ed-9d65-4928-8d3f-79984fc54670 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:52.814608 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae34ed-9d65-4928-8d3f-79984fc54670 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:52.826181 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ae34ed-9d65-4928-8d3f-79984fc54670 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:05:52.834568 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-63ae34ed-9d65-4928-8d3f-79984fc54670 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:05:52.834568 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:05:52.834568 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:05:52.834568 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:05:52.834568 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:05:52.834568 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:05:52.834568 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:05:52.834568 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:05:52.834568 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:05:52.834568 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:05:52.834568 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:05:52.834568 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:05:52.834568 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:05:52.834568 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:05:52.834568 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:05:52.834568 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:05:52.835945 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:05:52.835945 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:05:52.835945 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:05:52.835945 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:05:52.835945 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:05:52.835945 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:05:52.835945 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:05:52.835945 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:05:52.835945 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:05:52.835945 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:05:52.835945 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:05:52.835945 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:05:52.835945 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:05:52.835945 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:05:52.835945 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:05:52.835945 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:05:52.846323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:05:52.846323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:05:52.846323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:05:52.846323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:05:52.846323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:05:52.846323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:05:52.846323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:05:52.846323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:05:52.846323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:05:52.846323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:05:52.846323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 16:05:52.846323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:05:52.846323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 607/1235] 10.222.0.41 () {56 vars in 1102 bytes} [Fri Jun 3 16:05:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:05:52.847311 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a5efa-5395-4831-b839-c3f8633ded53 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:52.898557 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:52.960936 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a5efa-5395-4831-b839-c3f8633ded53 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:53.003027 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e68d15-19b1-47d6-97a5-8d941170e1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:53.003811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e68d15-19b1-47d6-97a5-8d941170e1ee None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:53.004147 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e68d15-19b1-47d6-97a5-8d941170e1ee None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:53.004546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e68d15-19b1-47d6-97a5-8d941170e1ee None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:53.005482 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e68d15-19b1-47d6-97a5-8d941170e1ee None admin] RBAC: Adding query filter params (name=service) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:05:53.006496 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e68d15-19b1-47d6-97a5-8d941170e1ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:53.007966 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e68d15-19b1-47d6-97a5-8d941170e1ee None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:53.020573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 629/1236] 10.222.0.41 () {56 vars in 1103 bytes} [Fri Jun 3 16:05:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:05:53.026814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae34ed-9d65-4928-8d3f-79984fc54670 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:53.127379 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:53.146661 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae34ed-9d65-4928-8d3f-79984fc54670 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:53.181730 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427037fd-b290-4efb-b960-f94ab5466666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:53.182786 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427037fd-b290-4efb-b960-f94ab5466666 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:53.183357 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427037fd-b290-4efb-b960-f94ab5466666 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:53.183802 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427037fd-b290-4efb-b960-f94ab5466666 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:53.196582 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427037fd-b290-4efb-b960-f94ab5466666 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:53.198759 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427037fd-b290-4efb-b960-f94ab5466666 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:53.218705 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-427037fd-b290-4efb-b960-f94ab5466666 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:05:53.218705 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:05:53.218705 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:05:53.218705 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:05:53.218705 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:05:53.218705 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:05:53.218705 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:05:53.218705 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:05:53.218705 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:05:53.218705 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:05:53.218705 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:05:53.218705 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:05:53.218705 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:05:53.218705 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:05:53.218705 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:05:53.219926 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:05:53.219926 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:05:53.219926 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:05:53.219926 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:05:53.219926 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:05:53.219926 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:05:53.219926 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:05:53.219926 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:05:53.219926 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:05:53.219926 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:05:53.219926 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:05:53.219926 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:05:53.219926 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:05:53.219926 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:05:53.219926 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:05:53.219926 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:05:53.219926 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:05:53.220994 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:05:53.220994 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:05:53.220994 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:05:53.220994 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:05:53.222873 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 608/1237] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:05:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:05:53.228824 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e68d15-19b1-47d6-97a5-8d941170e1ee None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:53.344596 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e68d15-19b1-47d6-97a5-8d941170e1ee None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:53.354788 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:53.384717 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e78038-32de-498f-9667-0763b62ab81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:53.385312 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e78038-32de-498f-9667-0763b62ab81b None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:53.385600 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e78038-32de-498f-9667-0763b62ab81b None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:53.385879 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e78038-32de-498f-9667-0763b62ab81b None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:53.386603 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e78038-32de-498f-9667-0763b62ab81b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:05:53.386914 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e78038-32de-498f-9667-0763b62ab81b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:53.388817 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e78038-32de-498f-9667-0763b62ab81b None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:53.400744 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 630/1238] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:05:53.406547 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427037fd-b290-4efb-b960-f94ab5466666 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:53.520057 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427037fd-b290-4efb-b960-f94ab5466666 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:53.557847 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d04213f-f316-4704-99a0-5bce7ec63b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:53.558719 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04213f-f316-4704-99a0-5bce7ec63b14 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:53.559051 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04213f-f316-4704-99a0-5bce7ec63b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:53.559380 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04213f-f316-4704-99a0-5bce7ec63b14 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:53.562205 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:53.567496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d04213f-f316-4704-99a0-5bce7ec63b14 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:53.569138 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d04213f-f316-4704-99a0-5bce7ec63b14 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:53.585551 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-0d04213f-f316-4704-99a0-5bce7ec63b14 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 03 16:05:53.585551 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:05:53.585551 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:05:53.585551 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:05:53.585551 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:05:53.585551 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:05:53.585551 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:05:53.585551 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:05:53.585551 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:05:53.585551 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:05:53.585551 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:05:53.585551 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:05:53.585551 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:05:53.585551 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:05:53.585551 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:05:53.585551 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:05:53.586788 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:05:53.586788 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:05:53.586788 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:05:53.586788 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:05:53.586788 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:05:53.586788 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:05:53.586788 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:05:53.586788 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:05:53.586788 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:05:53.586788 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:05:53.586788 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:05:53.586788 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:05:53.586788 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:05:53.586788 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:05:53.586788 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:05:53.586788 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:05:53.586788 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:05:53.586788 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:05:53.588196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:05:53.588196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:05:53.588196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:05:53.588196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:05:53.588196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:05:53.588196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:05:53.588196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:05:53.588196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:05:53.588196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 03 16:05:53.588196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:05:53.588196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 609/1239] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:05:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 16:05:53.593858 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e78038-32de-498f-9667-0763b62ab81b None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:53.706711 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e78038-32de-498f-9667-0763b62ab81b None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:53.742753 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0132ec-c0df-46a7-b077-3e3ee9830c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:53.743497 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0132ec-c0df-46a7-b077-3e3ee9830c5b None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:53.743787 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0132ec-c0df-46a7-b077-3e3ee9830c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:53.744065 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0132ec-c0df-46a7-b077-3e3ee9830c5b None admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:53.745053 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0132ec-c0df-46a7-b077-3e3ee9830c5b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:05:53.745399 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0132ec-c0df-46a7-b077-3e3ee9830c5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:53.747480 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0132ec-c0df-46a7-b077-3e3ee9830c5b None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:53.766747 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 631/1240] 10.222.0.41 () {56 vars in 1143 bytes} [Fri Jun 3 16:05:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:05:53.772922 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d04213f-f316-4704-99a0-5bce7ec63b14 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:53.889148 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d04213f-f316-4704-99a0-5bce7ec63b14 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:53.935711 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887aff8c-8bf4-4ebd-a6e8-9443e32c7a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:53.936716 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887aff8c-8bf4-4ebd-a6e8-9443e32c7a86 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:53.937075 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887aff8c-8bf4-4ebd-a6e8-9443e32c7a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:53.937363 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887aff8c-8bf4-4ebd-a6e8-9443e32c7a86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:54.006355 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:54.019894 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887aff8c-8bf4-4ebd-a6e8-9443e32c7a86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:54.021684 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887aff8c-8bf4-4ebd-a6e8-9443e32c7a86 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:54.148425 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 610/1241] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:05:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 16:05:54.155041 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0132ec-c0df-46a7-b077-3e3ee9830c5b None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:54.186257 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:54.273582 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0132ec-c0df-46a7-b077-3e3ee9830c5b None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:54.313207 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d233df-12d3-4f2d-a506-ccfc6dc0904e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:54.314093 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d233df-12d3-4f2d-a506-ccfc6dc0904e None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:54.314287 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d233df-12d3-4f2d-a506-ccfc6dc0904e None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:54.314560 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d233df-12d3-4f2d-a506-ccfc6dc0904e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:54.329630 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d233df-12d3-4f2d-a506-ccfc6dc0904e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:54.331361 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d233df-12d3-4f2d-a506-ccfc6dc0904e None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:54.356292 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-b0d233df-12d3-4f2d-a506-ccfc6dc0904e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:05:54.356292 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:05:54.356292 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:05:54.356292 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:05:54.356292 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:05:54.356292 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:05:54.356292 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:05:54.356292 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:05:54.356292 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:05:54.356292 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:05:54.356292 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:05:54.356292 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:05:54.356292 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:05:54.356292 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:05:54.356292 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:05:54.357563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:05:54.357563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:05:54.357563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:05:54.357563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:05:54.357563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:05:54.357563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:05:54.357563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:05:54.357563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:05:54.357563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:05:54.357563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:05:54.357563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:05:54.357563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:05:54.357563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:05:54.357563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:05:54.357563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:05:54.357563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:05:54.357563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:05:54.358590 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:05:54.358590 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:05:54.358590 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:05:54.358590 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:05:54.358590 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 632/1242] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:05:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:05:54.364924 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887aff8c-8bf4-4ebd-a6e8-9443e32c7a86 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:54.478443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:54.484509 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887aff8c-8bf4-4ebd-a6e8-9443e32c7a86 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:54.523331 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51957088-5b7e-445e-9292-5866cbd6dee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:54.524395 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51957088-5b7e-445e-9292-5866cbd6dee2 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:54.524575 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51957088-5b7e-445e-9292-5866cbd6dee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:54.524794 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51957088-5b7e-445e-9292-5866cbd6dee2 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:54.525515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51957088-5b7e-445e-9292-5866cbd6dee2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:05:54.525823 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51957088-5b7e-445e-9292-5866cbd6dee2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:54.527270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51957088-5b7e-445e-9292-5866cbd6dee2 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:54.540692 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 611/1243] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:05:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:05:54.547972 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d233df-12d3-4f2d-a506-ccfc6dc0904e None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:54.647329 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:54.675748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d233df-12d3-4f2d-a506-ccfc6dc0904e None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:54.717639 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981473e9-0bf1-4df3-b3a5-aee52ebf3aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:54.718513 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981473e9-0bf1-4df3-b3a5-aee52ebf3aaf None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:54.718852 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981473e9-0bf1-4df3-b3a5-aee52ebf3aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:54.719259 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981473e9-0bf1-4df3-b3a5-aee52ebf3aaf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:54.726610 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981473e9-0bf1-4df3-b3a5-aee52ebf3aaf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:54.728152 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981473e9-0bf1-4df3-b3a5-aee52ebf3aaf None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:54.744432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-981473e9-0bf1-4df3-b3a5-aee52ebf3aaf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:05:54.744432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:05:54.744432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:05:54.744432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:05:54.744432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:05:54.744432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:05:54.744432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:05:54.744432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:05:54.744432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:05:54.744432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:05:54.744432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:05:54.744432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:05:54.744432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:05:54.744432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:05:54.744432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:05:54.745558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:05:54.745558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:05:54.745558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:05:54.745558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:05:54.745558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:05:54.745558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:05:54.745558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:05:54.745558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:05:54.745558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:05:54.745558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:05:54.745558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:05:54.745558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:05:54.745558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:05:54.745558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:05:54.745558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:05:54.745558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:05:54.745558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:05:54.746776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:05:54.746776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:05:54.746776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:05:54.746776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:05:54.746776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:05:54.746776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:05:54.746776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:05:54.746776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:05:54.746776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 633/1244] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:05:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:05:54.752057 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51957088-5b7e-445e-9292-5866cbd6dee2 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:54.865380 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51957088-5b7e-445e-9292-5866cbd6dee2 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:54.904241 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292bc6bc-c253-44c5-a994-f01ec28ba871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:54.905030 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bc6bc-c253-44c5-a994-f01ec28ba871 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:54.905322 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bc6bc-c253-44c5-a994-f01ec28ba871 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:54.905596 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bc6bc-c253-44c5-a994-f01ec28ba871 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:54.906602 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292bc6bc-c253-44c5-a994-f01ec28ba871 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:05:54.906944 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292bc6bc-c253-44c5-a994-f01ec28ba871 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:54.908650 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292bc6bc-c253-44c5-a994-f01ec28ba871 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:54.931193 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 612/1245] 10.222.0.41 () {56 vars in 1148 bytes} [Fri Jun 3 16:05:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:05:54.936648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981473e9-0bf1-4df3-b3a5-aee52ebf3aaf None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:55.062896 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981473e9-0bf1-4df3-b3a5-aee52ebf3aaf None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:55.102794 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc635a2-b1c9-4f97-8430-96482a84fbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:55.103501 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc635a2-b1c9-4f97-8430-96482a84fbed None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:55.103787 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc635a2-b1c9-4f97-8430-96482a84fbed None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:55.104060 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc635a2-b1c9-4f97-8430-96482a84fbed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:55.104913 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc635a2-b1c9-4f97-8430-96482a84fbed None admin] RBAC: Adding query filter params (role.id=fe695aedbe8249a28c992a19264bc34b, scope.project.id=c274254e510a468e9acbecc6298301cc, user.id=be901786f8934d559e0bac5d622f97cb) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:05:55.105228 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc635a2-b1c9-4f97-8430-96482a84fbed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:55.107180 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc635a2-b1c9-4f97-8430-96482a84fbed None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:55.113926 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 634/1246] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:05:54 2022] GET /identity/v3/role_assignments?user.id=be901786f8934d559e0bac5d622f97cb&scope.project.id=c274254e510a468e9acbecc6298301cc&role.id=fe695aedbe8249a28c992a19264bc34b => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:05:57.449131 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abb41a-60e2-468c-aa15-253fa96b421a None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:57.450427 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abb41a-60e2-468c-aa15-253fa96b421a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:57.450427 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abb41a-60e2-468c-aa15-253fa96b421a None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:57.451633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 613/1247] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:05:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:05:57.457109 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f6633-96d5-4038-9951-d36ee81d918d None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:57.457563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f6633-96d5-4038-9951-d36ee81d918d None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:57.457655 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f6633-96d5-4038-9951-d36ee81d918d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:57.462733 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:57.475583 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-9f3f6633-96d5-4038-9951-d36ee81d918d None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:05:57.476785 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-9f3f6633-96d5-4038-9951-d36ee81d918d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:05:57.549532 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 635/1248] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:05:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:05:57.572562 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46abb41a-60e2-468c-aa15-253fa96b421a None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:57.591733 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:57.686567 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46abb41a-60e2-468c-aa15-253fa96b421a None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:57.724506 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8811a9-e0e9-4518-83e5-429378df3d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:57.725297 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8811a9-e0e9-4518-83e5-429378df3d32 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:57.725575 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8811a9-e0e9-4518-83e5-429378df3d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:57.725911 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8811a9-e0e9-4518-83e5-429378df3d32 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:57.737052 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8811a9-e0e9-4518-83e5-429378df3d32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:05:57.745482 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-3a8811a9-e0e9-4518-83e5-429378df3d32 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:05:57.745482 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:05:57.745482 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:05:57.745482 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:05:57.745482 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:05:57.745482 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:05:57.745482 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:05:57.745482 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:05:57.745482 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:05:57.745482 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:05:57.745482 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:05:57.745482 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:05:57.745482 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:05:57.745482 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:05:57.745482 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:05:57.745482 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:05:57.747115 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:05:57.747115 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:05:57.747115 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:05:57.747115 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:05:57.747115 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:05:57.747115 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:05:57.747115 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:05:57.747115 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:05:57.747115 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:05:57.747115 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:05:57.747115 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:05:57.747115 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:05:57.747115 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:05:57.747115 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:05:57.747115 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:05:57.747115 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:05:57.753229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:05:57.753229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:05:57.753229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:05:57.753229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:05:57.753229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:05:57.753229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:05:57.753229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:05:57.753229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:05:57.753229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:05:57.753229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:05:57.753229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:05:57.753229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:05:57.753229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 614/1249] 10.222.0.41 () {56 vars in 1096 bytes} [Fri Jun 3 16:05:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:05:57.754090 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f6633-96d5-4038-9951-d36ee81d918d None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:57.860095 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f6633-96d5-4038-9951-d36ee81d918d None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:57.889570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:57.895137 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d77280b-35e5-4c7c-90de-9049c4965a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:57.895844 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77280b-35e5-4c7c-90de-9049c4965a4d None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:57.896169 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77280b-35e5-4c7c-90de-9049c4965a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:57.896447 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77280b-35e5-4c7c-90de-9049c4965a4d None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:57.897196 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d77280b-35e5-4c7c-90de-9049c4965a4d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:05:57.897507 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d77280b-35e5-4c7c-90de-9049c4965a4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:57.899250 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d77280b-35e5-4c7c-90de-9049c4965a4d None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:57.908543 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 636/1250] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:05:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:05:57.914201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8811a9-e0e9-4518-83e5-429378df3d32 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:57.996179 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:58.024343 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8811a9-e0e9-4518-83e5-429378df3d32 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:58.063321 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc99e6-dc87-4e1d-abae-4db666b3c0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:58.064232 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc99e6-dc87-4e1d-abae-4db666b3c0cc None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:58.064442 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc99e6-dc87-4e1d-abae-4db666b3c0cc None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:58.064693 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc99e6-dc87-4e1d-abae-4db666b3c0cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:58.076236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbc99e6-dc87-4e1d-abae-4db666b3c0cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:58.077791 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbc99e6-dc87-4e1d-abae-4db666b3c0cc None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:58.095770 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-8bbc99e6-dc87-4e1d-abae-4db666b3c0cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:05:58.095770 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:05:58.095770 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:05:58.095770 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:05:58.095770 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:05:58.095770 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:05:58.095770 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:05:58.095770 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:05:58.095770 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:05:58.095770 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:05:58.095770 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:05:58.095770 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:05:58.095770 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:05:58.095770 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:05:58.095770 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:05:58.096876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:05:58.096876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:05:58.096876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:05:58.096876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:05:58.096876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:05:58.096876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:05:58.096876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:05:58.096876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:05:58.096876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:05:58.096876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:05:58.096876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:05:58.096876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:05:58.096876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:05:58.096876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:05:58.096876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:05:58.096876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:05:58.096876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:05:58.097975 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:05:58.097975 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:05:58.097975 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:05:58.097975 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:05:58.097975 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 615/1251] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:05:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:05:58.102780 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d77280b-35e5-4c7c-90de-9049c4965a4d None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:58.211322 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d77280b-35e5-4c7c-90de-9049c4965a4d None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:58.254750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6142cd-f7f7-4f2b-8795-af473c346ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:58.255510 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6142cd-f7f7-4f2b-8795-af473c346ccd None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:58.255801 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6142cd-f7f7-4f2b-8795-af473c346ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:58.256082 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6142cd-f7f7-4f2b-8795-af473c346ccd None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:58.256834 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6142cd-f7f7-4f2b-8795-af473c346ccd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:05:58.257155 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6142cd-f7f7-4f2b-8795-af473c346ccd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:58.258573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6142cd-f7f7-4f2b-8795-af473c346ccd None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:58.271861 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 637/1252] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:05:58.278542 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc99e6-dc87-4e1d-abae-4db666b3c0cc None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:58.384759 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc99e6-dc87-4e1d-abae-4db666b3c0cc None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:58.412581 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:58.422238 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dff950-fe33-4b24-85bf-06ad65498195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:58.423107 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dff950-fe33-4b24-85bf-06ad65498195 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:58.423424 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dff950-fe33-4b24-85bf-06ad65498195 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:58.423703 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dff950-fe33-4b24-85bf-06ad65498195 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:58.432264 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dff950-fe33-4b24-85bf-06ad65498195 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:58.433741 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dff950-fe33-4b24-85bf-06ad65498195 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:58.449010 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-b7dff950-fe33-4b24-85bf-06ad65498195 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 03 16:05:58.449010 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:05:58.449010 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:05:58.449010 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:05:58.449010 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:05:58.449010 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:05:58.449010 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:05:58.449010 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:05:58.449010 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:05:58.449010 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:05:58.449010 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:05:58.449010 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:05:58.449010 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:05:58.449010 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:05:58.449010 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:05:58.449010 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:05:58.450603 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:05:58.450603 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:05:58.450603 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:05:58.450603 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:05:58.450603 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:05:58.450603 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:05:58.450603 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:05:58.450603 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:05:58.450603 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:05:58.450603 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:05:58.450603 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:05:58.450603 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:05:58.450603 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:05:58.450603 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:05:58.450603 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:05:58.450603 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:05:58.450603 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:05:58.450603 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:05:58.451924 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:05:58.451924 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:05:58.451924 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:05:58.451924 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:05:58.451924 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:05:58.451924 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:05:58.451924 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:05:58.451924 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:05:58.451924 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 03 16:05:58.451924 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:05:58.451924 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 616/1253] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:05:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:05:58.456325 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6142cd-f7f7-4f2b-8795-af473c346ccd None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:58.513485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:58.561879 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6142cd-f7f7-4f2b-8795-af473c346ccd None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:58.597785 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a818a972-acc8-43de-9dcf-96c02543eaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:58.598472 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818a972-acc8-43de-9dcf-96c02543eaa0 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:58.598756 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818a972-acc8-43de-9dcf-96c02543eaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:58.599033 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818a972-acc8-43de-9dcf-96c02543eaa0 None admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:58.599929 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a818a972-acc8-43de-9dcf-96c02543eaa0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:05:58.600248 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a818a972-acc8-43de-9dcf-96c02543eaa0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:58.601718 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a818a972-acc8-43de-9dcf-96c02543eaa0 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:58.625516 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 638/1254] 10.222.0.41 () {56 vars in 1143 bytes} [Fri Jun 3 16:05:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:05:58.626734 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dff950-fe33-4b24-85bf-06ad65498195 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:58.728175 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dff950-fe33-4b24-85bf-06ad65498195 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:58.760768 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2de6b-92fd-4084-8a65-04deccaf57f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:58.761400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2de6b-92fd-4084-8a65-04deccaf57f1 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:58.762179 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2de6b-92fd-4084-8a65-04deccaf57f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:58.762446 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2de6b-92fd-4084-8a65-04deccaf57f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:58.817761 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:58.833678 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2de6b-92fd-4084-8a65-04deccaf57f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:58.835125 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2de6b-92fd-4084-8a65-04deccaf57f1 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:58.934177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 617/1255] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:05:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 16:05:58.939998 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a818a972-acc8-43de-9dcf-96c02543eaa0 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:59.060346 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a818a972-acc8-43de-9dcf-96c02543eaa0 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:59.075980 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:59.097014 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31064c-4d00-4d36-8aad-e29c64281811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:59.097808 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31064c-4d00-4d36-8aad-e29c64281811 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:59.098092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31064c-4d00-4d36-8aad-e29c64281811 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:59.098372 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31064c-4d00-4d36-8aad-e29c64281811 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:59.108125 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c31064c-4d00-4d36-8aad-e29c64281811 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:59.109554 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c31064c-4d00-4d36-8aad-e29c64281811 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:59.126923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-2c31064c-4d00-4d36-8aad-e29c64281811 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:05:59.126923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:05:59.126923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:05:59.126923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:05:59.126923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:05:59.126923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:05:59.126923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:05:59.126923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:05:59.126923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:05:59.126923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:05:59.126923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:05:59.126923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:05:59.126923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:05:59.126923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:05:59.126923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:05:59.127975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:05:59.127975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:05:59.127975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:05:59.127975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:05:59.127975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:05:59.127975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:05:59.127975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:05:59.127975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:05:59.127975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:05:59.127975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:05:59.127975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:05:59.127975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:05:59.127975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:05:59.127975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:05:59.127975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:05:59.127975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:05:59.127975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:05:59.128918 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:05:59.128918 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:05:59.128918 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:05:59.128918 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:05:59.128918 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 639/1256] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:05:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:05:59.133759 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2de6b-92fd-4084-8a65-04deccaf57f1 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:59.217452 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:59.231622 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2de6b-92fd-4084-8a65-04deccaf57f1 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:59.265133 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba22221-96a4-4d8e-a8d4-f3134f27e568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:59.265781 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba22221-96a4-4d8e-a8d4-f3134f27e568 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:59.266062 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba22221-96a4-4d8e-a8d4-f3134f27e568 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:59.266334 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba22221-96a4-4d8e-a8d4-f3134f27e568 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:59.267138 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba22221-96a4-4d8e-a8d4-f3134f27e568 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:05:59.267488 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba22221-96a4-4d8e-a8d4-f3134f27e568 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:59.268888 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba22221-96a4-4d8e-a8d4-f3134f27e568 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:59.279911 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 618/1257] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:05:59.285569 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31064c-4d00-4d36-8aad-e29c64281811 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:59.384727 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31064c-4d00-4d36-8aad-e29c64281811 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:59.417061 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5861e-311d-4ce0-baf9-06dd312a4e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:59.417846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5861e-311d-4ce0-baf9-06dd312a4e11 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:59.418648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5861e-311d-4ce0-baf9-06dd312a4e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:59.418926 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5861e-311d-4ce0-baf9-06dd312a4e11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:59.425369 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c5861e-311d-4ce0-baf9-06dd312a4e11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:59.426797 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c5861e-311d-4ce0-baf9-06dd312a4e11 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:59.440949 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-a7c5861e-311d-4ce0-baf9-06dd312a4e11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:05:59.440949 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:05:59.440949 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:05:59.440949 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:05:59.440949 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:05:59.440949 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:05:59.440949 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:05:59.440949 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:05:59.440949 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:05:59.440949 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:05:59.440949 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:05:59.440949 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:05:59.440949 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:05:59.440949 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:05:59.440949 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:05:59.442020 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:05:59.442020 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:05:59.442020 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:05:59.442020 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:05:59.442020 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:05:59.442020 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:05:59.442020 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:05:59.442020 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:05:59.442020 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:05:59.442020 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:05:59.442020 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:05:59.442020 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:05:59.442020 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:05:59.442020 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:05:59.442020 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:05:59.442020 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:05:59.442020 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:05:59.442945 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:05:59.442945 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:05:59.442945 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:05:59.442945 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:05:59.442945 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:05:59.442945 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:05:59.442945 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:05:59.442945 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:05:59.442945 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 640/1258] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:05:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:05:59.448081 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba22221-96a4-4d8e-a8d4-f3134f27e568 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:59.543162 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba22221-96a4-4d8e-a8d4-f3134f27e568 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:59.574053 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6270af76-b5b2-4165-85a7-c23bbee04d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:59.575409 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6270af76-b5b2-4165-85a7-c23bbee04d1c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:59.575692 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6270af76-b5b2-4165-85a7-c23bbee04d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:59.575962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6270af76-b5b2-4165-85a7-c23bbee04d1c None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:59.576735 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6270af76-b5b2-4165-85a7-c23bbee04d1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:05:59.577041 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6270af76-b5b2-4165-85a7-c23bbee04d1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:59.578450 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6270af76-b5b2-4165-85a7-c23bbee04d1c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:59.590511 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 619/1259] 10.222.0.41 () {56 vars in 1148 bytes} [Fri Jun 3 16:05:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:05:59.596317 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5861e-311d-4ce0-baf9-06dd312a4e11 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:05:59.620986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:05:59.692696 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5861e-311d-4ce0-baf9-06dd312a4e11 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:05:59.726087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ee81d-b43d-44e7-9364-fee9cf635148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:05:59.726734 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ee81d-b43d-44e7-9364-fee9cf635148 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:05:59.727021 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ee81d-b43d-44e7-9364-fee9cf635148 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:05:59.727320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ee81d-b43d-44e7-9364-fee9cf635148 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:05:59.728156 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ee81d-b43d-44e7-9364-fee9cf635148 None admin] RBAC: Adding query filter params (role.id=75936a608b8149099ff54e317f3a6740, scope.project.id=c274254e510a468e9acbecc6298301cc, user.id=be901786f8934d559e0bac5d622f97cb) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:05:59.728473 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ee81d-b43d-44e7-9364-fee9cf635148 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:05:59.729883 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ee81d-b43d-44e7-9364-fee9cf635148 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:05:59.735169 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 641/1260] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:05:59 2022] GET /identity/v3/role_assignments?user.id=be901786f8934d559e0bac5d622f97cb&scope.project.id=c274254e510a468e9acbecc6298301cc&role.id=75936a608b8149099ff54e317f3a6740 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:06:01.915935 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea0257-def3-40e3-b412-63cb857eb12c None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:01.916459 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea0257-def3-40e3-b412-63cb857eb12c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:01.916577 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea0257-def3-40e3-b412-63cb857eb12c None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:01.917865 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 620/1261] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:06:01.922882 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06898e53-2c8a-45e7-83b2-264ae7c2a6c4 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:01.923309 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06898e53-2c8a-45e7-83b2-264ae7c2a6c4 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:01.923474 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06898e53-2c8a-45e7-83b2-264ae7c2a6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:01.939810 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-06898e53-2c8a-45e7-83b2-264ae7c2a6c4 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:06:01.940982 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-06898e53-2c8a-45e7-83b2-264ae7c2a6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:06:02.005315 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 642/1262] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:06:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:06:02.030787 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea0257-def3-40e3-b412-63cb857eb12c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:06:02.046204 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:06:02.156853 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea0257-def3-40e3-b412-63cb857eb12c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:06:02.188993 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ca19d-977d-4e5c-95d8-f525880bf39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:06:02.189715 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ca19d-977d-4e5c-95d8-f525880bf39a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:02.189999 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ca19d-977d-4e5c-95d8-f525880bf39a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:02.190277 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ca19d-977d-4e5c-95d8-f525880bf39a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:02.200617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ca19d-977d-4e5c-95d8-f525880bf39a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:06:02.208139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-f17ca19d-977d-4e5c-95d8-f525880bf39a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:06:02.208139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:06:02.208139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:06:02.208139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:06:02.208139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:06:02.208139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:06:02.208139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:06:02.208139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:06:02.208139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:06:02.208139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:06:02.208139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:06:02.208139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:06:02.208139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:06:02.208139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:06:02.208139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:06:02.208139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:06:02.209188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:06:02.209188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:06:02.209188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:06:02.209188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:06:02.209188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:06:02.209188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:06:02.209188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:06:02.209188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:06:02.209188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:06:02.209188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:06:02.209188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:06:02.209188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:06:02.209188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:06:02.209188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:06:02.209188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:06:02.209188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:06:02.210178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:06:02.210178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:06:02.210178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:06:02.210178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:06:02.210178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:06:02.210178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:06:02.210178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:06:02.210178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:06:02.210178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:06:02.210178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:06:02.210178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:06:02.210178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:06:02.210178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 621/1263] 10.222.0.41 () {56 vars in 1096 bytes} [Fri Jun 3 16:06:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:06:02.215340 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06898e53-2c8a-45e7-83b2-264ae7c2a6c4 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:06:02.232914 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:06:02.310557 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06898e53-2c8a-45e7-83b2-264ae7c2a6c4 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:06:02.343383 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8273b3-182a-48c7-a30c-a1d92507d312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:06:02.344033 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8273b3-182a-48c7-a30c-a1d92507d312 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:02.344316 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8273b3-182a-48c7-a30c-a1d92507d312 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:02.344589 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8273b3-182a-48c7-a30c-a1d92507d312 None admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:02.345320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8273b3-182a-48c7-a30c-a1d92507d312 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:06:02.345622 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8273b3-182a-48c7-a30c-a1d92507d312 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:06:02.346984 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8273b3-182a-48c7-a30c-a1d92507d312 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:06:02.354595 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 643/1264] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:06:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:06:02.361692 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ca19d-977d-4e5c-95d8-f525880bf39a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:06:02.425640 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:06:02.459101 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ca19d-977d-4e5c-95d8-f525880bf39a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:06:02.491080 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd1b1f-dedd-4898-8374-1f8d4a4cae96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:06:02.491874 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd1b1f-dedd-4898-8374-1f8d4a4cae96 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:02.492162 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd1b1f-dedd-4898-8374-1f8d4a4cae96 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:02.492439 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd1b1f-dedd-4898-8374-1f8d4a4cae96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:02.503151 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd1b1f-dedd-4898-8374-1f8d4a4cae96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:06:02.504621 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd1b1f-dedd-4898-8374-1f8d4a4cae96 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:06:02.521108 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-f5cd1b1f-dedd-4898-8374-1f8d4a4cae96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:06:02.521108 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:06:02.521108 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:06:02.521108 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:06:02.521108 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:06:02.521108 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:06:02.521108 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:06:02.521108 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:06:02.521108 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:06:02.521108 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:06:02.521108 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:06:02.521108 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:06:02.521108 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:06:02.521108 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:06:02.521108 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:06:02.522131 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:06:02.522131 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:06:02.522131 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:06:02.522131 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:06:02.522131 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:06:02.522131 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:06:02.522131 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:06:02.522131 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:06:02.522131 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:06:02.522131 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:06:02.522131 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:06:02.522131 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:06:02.522131 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:06:02.522131 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:06:02.522131 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:06:02.522131 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:06:02.522131 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:06:02.523396 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:06:02.523396 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:06:02.523396 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:06:02.523396 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:06:02.523396 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 622/1265] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:06:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:06:02.528186 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8273b3-182a-48c7-a30c-a1d92507d312 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:06:02.605036 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:06:02.624576 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8273b3-182a-48c7-a30c-a1d92507d312 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:06:02.656283 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7eb292-9a07-47a4-a7a8-0fe9fbd87868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:06:02.656930 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7eb292-9a07-47a4-a7a8-0fe9fbd87868 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:02.657213 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7eb292-9a07-47a4-a7a8-0fe9fbd87868 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:02.657489 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7eb292-9a07-47a4-a7a8-0fe9fbd87868 None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:02.658196 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7eb292-9a07-47a4-a7a8-0fe9fbd87868 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:06:02.658502 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7eb292-9a07-47a4-a7a8-0fe9fbd87868 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:06:02.659875 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7eb292-9a07-47a4-a7a8-0fe9fbd87868 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:06:02.671515 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 644/1266] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:06:02.676880 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd1b1f-dedd-4898-8374-1f8d4a4cae96 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:06:02.772349 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd1b1f-dedd-4898-8374-1f8d4a4cae96 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:06:02.791364 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:06:02.805333 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698e2949-71dd-4037-a2b7-8e79106610cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:06:02.806106 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e2949-71dd-4037-a2b7-8e79106610cf None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:02.806389 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e2949-71dd-4037-a2b7-8e79106610cf None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:02.806666 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e2949-71dd-4037-a2b7-8e79106610cf None admin] PATH_INFO: `/v3/users/placement` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:02.813812 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698e2949-71dd-4037-a2b7-8e79106610cf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:06:02.815237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698e2949-71dd-4037-a2b7-8e79106610cf None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:06:02.828982 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-698e2949-71dd-4037-a2b7-8e79106610cf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 03 16:06:02.828982 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:06:02.828982 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:06:02.828982 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:06:02.828982 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:06:02.828982 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:06:02.828982 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:06:02.828982 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:06:02.828982 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:06:02.828982 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:06:02.828982 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:06:02.828982 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:06:02.828982 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:06:02.828982 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:06:02.828982 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:06:02.828982 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:06:02.830036 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:06:02.830036 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:06:02.830036 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:06:02.830036 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:06:02.830036 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:06:02.830036 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:06:02.830036 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:06:02.830036 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:06:02.830036 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:06:02.830036 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:06:02.830036 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:06:02.830036 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:06:02.830036 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:06:02.830036 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:06:02.830036 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:06:02.830036 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:06:02.830036 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:06:02.830036 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:06:02.831165 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:06:02.831165 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:06:02.831165 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:06:02.831165 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:06:02.831165 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:06:02.831165 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:06:02.831165 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:06:02.831165 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:06:02.831165 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 03 16:06:02.831165 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:06:02.831165 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 623/1267] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:06:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:06:02.835742 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7eb292-9a07-47a4-a7a8-0fe9fbd87868 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:06:02.931601 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7eb292-9a07-47a4-a7a8-0fe9fbd87868 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:06:02.963437 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e92d23-ef51-4218-b96f-7a25ca2c028a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:06:02.964085 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e92d23-ef51-4218-b96f-7a25ca2c028a None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:02.964387 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e92d23-ef51-4218-b96f-7a25ca2c028a None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:02.965174 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e92d23-ef51-4218-b96f-7a25ca2c028a None admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:02.965995 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e92d23-ef51-4218-b96f-7a25ca2c028a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:06:02.966304 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e92d23-ef51-4218-b96f-7a25ca2c028a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:06:02.967734 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e92d23-ef51-4218-b96f-7a25ca2c028a None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:06:02.975996 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:06:02.986038 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 645/1268] 10.222.0.41 () {56 vars in 1143 bytes} [Fri Jun 3 16:06:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:06:02.991615 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698e2949-71dd-4037-a2b7-8e79106610cf None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:06:03.095521 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698e2949-71dd-4037-a2b7-8e79106610cf None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:06:03.127164 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1910531b-766a-4c51-bc5c-70ef9a589ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:06:03.127962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910531b-766a-4c51-bc5c-70ef9a589ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:03.128244 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910531b-766a-4c51-bc5c-70ef9a589ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:03.128516 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910531b-766a-4c51-bc5c-70ef9a589ef6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:03.139003 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1910531b-766a-4c51-bc5c-70ef9a589ef6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:06:03.140460 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1910531b-766a-4c51-bc5c-70ef9a589ef6 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:06:03.157617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-1910531b-766a-4c51-bc5c-70ef9a589ef6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:06:03.157617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:06:03.157617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:06:03.157617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:06:03.157617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:06:03.157617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:06:03.157617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:06:03.157617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:06:03.157617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:06:03.157617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:06:03.157617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:06:03.157617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:06:03.157617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:06:03.157617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:06:03.157617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:06:03.158678 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:06:03.158678 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:06:03.158678 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:06:03.158678 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:06:03.158678 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:06:03.158678 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:06:03.158678 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:06:03.158678 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:06:03.158678 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:06:03.158678 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:06:03.158678 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:06:03.158678 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:06:03.158678 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:06:03.158678 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:06:03.158678 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:06:03.158678 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:06:03.158678 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:06:03.159780 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:06:03.159780 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:06:03.159780 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:06:03.159780 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:06:03.159780 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 624/1269] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:06:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:06:03.164484 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e92d23-ef51-4218-b96f-7a25ca2c028a None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:06:03.259464 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e92d23-ef51-4218-b96f-7a25ca2c028a None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:06:03.292874 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d93a07-d6f1-4b11-97d3-ce53af8161df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:06:03.293525 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d93a07-d6f1-4b11-97d3-ce53af8161df None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:03.293805 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d93a07-d6f1-4b11-97d3-ce53af8161df None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:03.294081 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d93a07-d6f1-4b11-97d3-ce53af8161df None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:03.294788 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d93a07-d6f1-4b11-97d3-ce53af8161df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:06:03.295096 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d93a07-d6f1-4b11-97d3-ce53af8161df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:06:03.296475 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d93a07-d6f1-4b11-97d3-ce53af8161df None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:06:03.307168 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 646/1270] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:06:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:06:03.312696 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1910531b-766a-4c51-bc5c-70ef9a589ef6 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:06:03.343692 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:06:03.410517 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1910531b-766a-4c51-bc5c-70ef9a589ef6 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:06:03.442269 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537ff48-fa3e-4e3b-93a2-c8833beb242c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:06:03.443040 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537ff48-fa3e-4e3b-93a2-c8833beb242c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:03.443346 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537ff48-fa3e-4e3b-93a2-c8833beb242c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:03.443629 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537ff48-fa3e-4e3b-93a2-c8833beb242c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:03.450523 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537ff48-fa3e-4e3b-93a2-c8833beb242c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:06:03.451986 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537ff48-fa3e-4e3b-93a2-c8833beb242c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:06:03.466230 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-a537ff48-fa3e-4e3b-93a2-c8833beb242c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:06:03.466230 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:06:03.466230 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:06:03.466230 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:06:03.466230 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:06:03.466230 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:06:03.466230 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:06:03.466230 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:06:03.466230 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:06:03.466230 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:06:03.466230 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:06:03.466230 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:06:03.466230 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:06:03.466230 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:06:03.466230 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:06:03.467248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:06:03.467248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:06:03.467248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:06:03.467248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:06:03.467248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:06:03.467248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:06:03.467248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:06:03.467248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:06:03.467248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:06:03.467248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:06:03.467248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:06:03.467248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:06:03.467248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:06:03.467248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:06:03.467248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:06:03.467248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:06:03.467248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:06:03.468196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:06:03.468196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:06:03.468196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:06:03.468196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:06:03.468196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:06:03.468196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:06:03.468196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:06:03.468196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:06:03.468196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 625/1271] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:06:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:06:03.473269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d93a07-d6f1-4b11-97d3-ce53af8161df None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:06:03.529705 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:06:03.569607 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d93a07-d6f1-4b11-97d3-ce53af8161df None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:06:03.601583 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033760e-e6f0-4b63-9d3a-46ea9951b4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:06:03.602226 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033760e-e6f0-4b63-9d3a-46ea9951b4a0 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:03.602510 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033760e-e6f0-4b63-9d3a-46ea9951b4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:03.602786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033760e-e6f0-4b63-9d3a-46ea9951b4a0 None admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:03.603592 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7033760e-e6f0-4b63-9d3a-46ea9951b4a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:06:03.603903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7033760e-e6f0-4b63-9d3a-46ea9951b4a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:06:03.605349 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7033760e-e6f0-4b63-9d3a-46ea9951b4a0 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:06:03.617225 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 647/1272] 10.222.0.41 () {56 vars in 1148 bytes} [Fri Jun 3 16:06:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:06:03.623674 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537ff48-fa3e-4e3b-93a2-c8833beb242c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:06:03.712565 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:06:03.721590 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537ff48-fa3e-4e3b-93a2-c8833beb242c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:06:03.754293 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283cf8a-988b-4bd9-9b00-a73a53122d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:06:03.755163 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283cf8a-988b-4bd9-9b00-a73a53122d39 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:03.755480 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283cf8a-988b-4bd9-9b00-a73a53122d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:03.755763 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283cf8a-988b-4bd9-9b00-a73a53122d39 None admin] PATH_INFO: `/v3/projects/c274254e510a468e9acbecc6298301cc/users/be901786f8934d559e0bac5d622f97cb/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:03.757661 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283cf8a-988b-4bd9-9b00-a73a53122d39 None admin] RBAC: Authorizing `identity:create_grant(project_id=c274254e510a468e9acbecc6298301cc, user_id=be901786f8934d559e0bac5d622f97cb, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:06:03.759078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283cf8a-988b-4bd9-9b00-a73a53122d39 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:06:03.771166 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 626/1273] 10.222.0.41 () {58 vars in 1439 bytes} [Fri Jun 3 16:06:03 2022] PUT /identity/v3/projects/c274254e510a468e9acbecc6298301cc/users/be901786f8934d559e0bac5d622f97cb/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:06:05.977003 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b422b22-987c-4f41-bbda-5ec1e6c23553 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:05.977552 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b422b22-987c-4f41-bbda-5ec1e6c23553 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:05.977552 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b422b22-987c-4f41-bbda-5ec1e6c23553 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:05.978909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 648/1274] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:06:05.984043 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82166d99-086d-4c9c-b011-2c8a922d3fda None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:05.984412 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82166d99-086d-4c9c-b011-2c8a922d3fda None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:05.984604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82166d99-086d-4c9c-b011-2c8a922d3fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:06.001353 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-82166d99-086d-4c9c-b011-2c8a922d3fda None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:06:06.002553 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-82166d99-086d-4c9c-b011-2c8a922d3fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:06:06.089618 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 627/1275] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:06:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:06:06.117682 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b422b22-987c-4f41-bbda-5ec1e6c23553 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:06:06.214603 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b422b22-987c-4f41-bbda-5ec1e6c23553 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:06:06.231867 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:06:06.249042 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7331e-ef87-47b9-8744-1560069653d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:06:06.249836 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7331e-ef87-47b9-8744-1560069653d8 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:06.250119 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7331e-ef87-47b9-8744-1560069653d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:06.250396 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7331e-ef87-47b9-8744-1560069653d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:06.260339 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc7331e-ef87-47b9-8744-1560069653d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 16:06:06.267921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-3cc7331e-ef87-47b9-8744-1560069653d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:06:06.267921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:06:06.267921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:06:06.267921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:06:06.267921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:06:06.267921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:06:06.267921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:06:06.267921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:06:06.267921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:06:06.267921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:06:06.267921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:06:06.267921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:06:06.267921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 16:06:06.267921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 16:06:06.267921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 16:06:06.267921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 16:06:06.268993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 16:06:06.268993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 16:06:06.268993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 16:06:06.268993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 16:06:06.268993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:06:06.268993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:06:06.268993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:06:06.268993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:06:06.268993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:06:06.268993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:06:06.268993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:06:06.268993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:06:06.268993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:06:06.268993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:06:06.268993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:06:06.268993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:06:06.270028 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:06:06.270028 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:06:06.270028 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:06:06.270028 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:06:06.270028 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 16:06:06.270028 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 16:06:06.270028 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 16:06:06.270028 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 16:06:06.270028 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 16:06:06.270028 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 16:06:06.270028 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 16:06:06.270028 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:06:06.270028 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 649/1276] 10.222.0.41 () {56 vars in 1096 bytes} [Fri Jun 3 16:06:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:06:06.275225 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82166d99-086d-4c9c-b011-2c8a922d3fda None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:06:06.352781 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:06:06.372655 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82166d99-086d-4c9c-b011-2c8a922d3fda None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:06:06.404797 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f2404b-0122-4404-bc31-5f315d629877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:06:06.405449 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2404b-0122-4404-bc31-5f315d629877 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:06.405733 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2404b-0122-4404-bc31-5f315d629877 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:06.406010 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2404b-0122-4404-bc31-5f315d629877 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:06.406751 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f2404b-0122-4404-bc31-5f315d629877 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:06:06.407062 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f2404b-0122-4404-bc31-5f315d629877 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:06:06.408470 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f2404b-0122-4404-bc31-5f315d629877 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:06:06.416236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 628/1277] 10.222.0.41 () {56 vars in 1099 bytes} [Fri Jun 3 16:06:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:06:06.422258 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7331e-ef87-47b9-8744-1560069653d8 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:06:06.522914 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7331e-ef87-47b9-8744-1560069653d8 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:06:06.555609 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9191f94-6cb7-423f-940c-16389fcafe05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:06:06.556386 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9191f94-6cb7-423f-940c-16389fcafe05 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:06.556673 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9191f94-6cb7-423f-940c-16389fcafe05 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:06.556952 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9191f94-6cb7-423f-940c-16389fcafe05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:06.566994 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9191f94-6cb7-423f-940c-16389fcafe05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:06:06.568459 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9191f94-6cb7-423f-940c-16389fcafe05 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:06:06.584846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-b9191f94-6cb7-423f-940c-16389fcafe05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:06:06.584846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:06:06.584846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:06:06.584846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:06:06.584846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:06:06.584846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:06:06.584846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:06:06.584846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:06:06.584846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:06:06.584846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:06:06.584846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:06:06.584846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:06:06.584846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:06:06.584846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:06:06.584846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:06:06.585722 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:06:06.585722 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:06:06.585722 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:06:06.585722 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:06:06.585722 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:06:06.585722 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:06:06.585722 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:06:06.585722 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:06:06.585722 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:06:06.585722 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:06:06.585722 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:06:06.585722 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:06:06.585722 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:06:06.585722 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:06:06.585722 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:06:06.585722 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:06:06.585722 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:06:06.586640 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:06:06.586640 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:06:06.586640 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:06:06.586640 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:06:06.586640 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 650/1278] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:06:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:06:06.592269 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f2404b-0122-4404-bc31-5f315d629877 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:06:06.686602 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f2404b-0122-4404-bc31-5f315d629877 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:06:06.718043 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32ce67-60d8-41a6-8546-a0d7e14587c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:06:06.718675 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32ce67-60d8-41a6-8546-a0d7e14587c0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:06.718957 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32ce67-60d8-41a6-8546-a0d7e14587c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:06.719230 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32ce67-60d8-41a6-8546-a0d7e14587c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:06.719970 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b32ce67-60d8-41a6-8546-a0d7e14587c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:06:06.720274 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b32ce67-60d8-41a6-8546-a0d7e14587c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:06:06.721616 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b32ce67-60d8-41a6-8546-a0d7e14587c0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:06:06.733349 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 629/1279] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:06:06.738754 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9191f94-6cb7-423f-940c-16389fcafe05 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:06:06.768246 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:06:06.835371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9191f94-6cb7-423f-940c-16389fcafe05 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:06:06.867810 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bfa279-2d11-4830-83bb-dcd90d8306be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:06:06.868590 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfa279-2d11-4830-83bb-dcd90d8306be None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:06.868874 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfa279-2d11-4830-83bb-dcd90d8306be None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:06.869153 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfa279-2d11-4830-83bb-dcd90d8306be None admin] PATH_INFO: `/v3/users/placement` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:06.876292 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bfa279-2d11-4830-83bb-dcd90d8306be None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:06:06.877705 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bfa279-2d11-4830-83bb-dcd90d8306be None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:06:06.892085 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-e1bfa279-2d11-4830-83bb-dcd90d8306be None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 03 16:06:06.892085 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:06:06.892085 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:06:06.892085 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:06:06.892085 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:06:06.892085 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:06:06.892085 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:06:06.892085 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:06:06.892085 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:06:06.892085 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:06:06.892085 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:06:06.892085 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:06:06.892085 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 16:06:06.892085 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 16:06:06.892085 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 16:06:06.892085 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 16:06:06.893087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:06:06.893087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:06:06.893087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 16:06:06.893087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:06:06.893087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 16:06:06.893087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 16:06:06.893087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:06:06.893087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:06:06.893087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:06:06.893087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:06:06.893087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:06:06.893087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:06:06.893087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:06:06.893087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:06:06.893087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:06:06.893087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:06:06.893087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:06:06.893087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:06:06.894106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:06:06.894106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:06:06.894106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 16:06:06.894106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 16:06:06.894106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 16:06:06.894106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 16:06:06.894106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 16:06:06.894106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 16:06:06.894106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 03 16:06:06.894106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:06:06.894106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 651/1280] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:06:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:06:06.899088 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32ce67-60d8-41a6-8546-a0d7e14587c0 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:06:06.903306 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:06:06.993376 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32ce67-60d8-41a6-8546-a0d7e14587c0 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:06:07.031028 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c180b2-e8ec-4471-9a1a-280804510c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:06:07.031715 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c180b2-e8ec-4471-9a1a-280804510c1a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:07.032003 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c180b2-e8ec-4471-9a1a-280804510c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:07.032283 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c180b2-e8ec-4471-9a1a-280804510c1a None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:07.033111 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c180b2-e8ec-4471-9a1a-280804510c1a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:06:07.033424 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c180b2-e8ec-4471-9a1a-280804510c1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:06:07.034839 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c180b2-e8ec-4471-9a1a-280804510c1a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:06:07.051238 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 630/1281] 10.222.0.41 () {56 vars in 1143 bytes} [Fri Jun 3 16:06:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:06:07.056840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bfa279-2d11-4830-83bb-dcd90d8306be None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:06:07.151075 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:06:07.160915 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bfa279-2d11-4830-83bb-dcd90d8306be None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:06:07.192516 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730eeee-944d-409d-92ad-d9fb52187e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:06:07.193154 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730eeee-944d-409d-92ad-d9fb52187e50 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:07.193439 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730eeee-944d-409d-92ad-d9fb52187e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:07.193714 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730eeee-944d-409d-92ad-d9fb52187e50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:07.259794 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5730eeee-944d-409d-92ad-d9fb52187e50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:06:07.261299 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5730eeee-944d-409d-92ad-d9fb52187e50 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:06:07.346562 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:06:07.357315 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 652/1282] 10.222.0.41 () {58 vars in 1282 bytes} [Fri Jun 3 16:06:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 16:06:07.363019 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c180b2-e8ec-4471-9a1a-280804510c1a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:06:07.422809 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:06:07.459194 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c180b2-e8ec-4471-9a1a-280804510c1a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:06:07.492288 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f3cd6-cf0f-4957-b3fd-eb125da5793f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:06:07.493056 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f3cd6-cf0f-4957-b3fd-eb125da5793f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:07.493339 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f3cd6-cf0f-4957-b3fd-eb125da5793f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:07.493615 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f3cd6-cf0f-4957-b3fd-eb125da5793f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:07.503183 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205f3cd6-cf0f-4957-b3fd-eb125da5793f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:06:07.504644 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205f3cd6-cf0f-4957-b3fd-eb125da5793f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:06:07.521500 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-205f3cd6-cf0f-4957-b3fd-eb125da5793f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:06:07.521500 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:06:07.521500 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:06:07.521500 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:06:07.521500 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:06:07.521500 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:06:07.521500 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:06:07.521500 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:06:07.521500 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:06:07.521500 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:06:07.521500 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:06:07.521500 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:06:07.521500 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 16:06:07.521500 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 16:06:07.521500 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 16:06:07.522524 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 16:06:07.522524 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:06:07.522524 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:06:07.522524 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:06:07.522524 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:06:07.522524 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:06:07.522524 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:06:07.522524 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:06:07.522524 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:06:07.522524 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:06:07.522524 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:06:07.522524 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:06:07.522524 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:06:07.522524 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:06:07.522524 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:06:07.522524 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:06:07.522524 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:06:07.523984 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 16:06:07.523984 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 16:06:07.523984 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 16:06:07.523984 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:06:07.523984 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 631/1283] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:06:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:06:07.529049 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730eeee-944d-409d-92ad-d9fb52187e50 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:06:07.623688 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730eeee-944d-409d-92ad-d9fb52187e50 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:06:07.655687 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac58ed-4a2b-467e-a3de-1f3c1d5c0ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:06:07.656333 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac58ed-4a2b-467e-a3de-1f3c1d5c0ca4 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:07.656618 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac58ed-4a2b-467e-a3de-1f3c1d5c0ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:07.656894 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac58ed-4a2b-467e-a3de-1f3c1d5c0ca4 None admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:07.657606 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac58ed-4a2b-467e-a3de-1f3c1d5c0ca4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:06:07.657912 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac58ed-4a2b-467e-a3de-1f3c1d5c0ca4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:06:07.659261 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac58ed-4a2b-467e-a3de-1f3c1d5c0ca4 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:06:07.670493 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 653/1284] 10.222.0.41 () {56 vars in 1109 bytes} [Fri Jun 3 16:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:06:07.675812 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f3cd6-cf0f-4957-b3fd-eb125da5793f None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:06:07.769573 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f3cd6-cf0f-4957-b3fd-eb125da5793f None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:06:07.783838 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:06:07.803726 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d3aba-cb36-4d46-af2a-e8d82f0f461a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:06:07.804487 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d3aba-cb36-4d46-af2a-e8d82f0f461a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:07.804765 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d3aba-cb36-4d46-af2a-e8d82f0f461a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:07.805034 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d3aba-cb36-4d46-af2a-e8d82f0f461a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:07.811990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0d3aba-cb36-4d46-af2a-e8d82f0f461a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:06:07.813405 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0d3aba-cb36-4d46-af2a-e8d82f0f461a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:06:07.826656 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-ad0d3aba-cb36-4d46-af2a-e8d82f0f461a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:06:07.826656 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:06:07.826656 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:06:07.826656 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:06:07.826656 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:06:07.826656 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:06:07.826656 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:06:07.826656 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:06:07.826656 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:06:07.826656 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:06:07.826656 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:06:07.826656 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:06:07.826656 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 16:06:07.826656 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 16:06:07.826656 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 16:06:07.827704 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 16:06:07.827704 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:06:07.827704 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:06:07.827704 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:06:07.827704 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:06:07.827704 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:06:07.827704 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:06:07.827704 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:06:07.827704 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jun 03 16:06:07.827704 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:06:07.827704 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:06:07.827704 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:06:07.827704 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:06:07.827704 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:06:07.827704 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:06:07.827704 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:06:07.827704 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:06:07.828796 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 16:06:07.828796 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 16:06:07.828796 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 16:06:07.828796 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 16:06:07.828796 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 16:06:07.828796 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 16:06:07.828796 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 16:06:07.828796 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jun 03 16:06:07.828796 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 632/1285] 10.222.0.41 () {56 vars in 1111 bytes} [Fri Jun 3 16:06:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:06:07.833452 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac58ed-4a2b-467e-a3de-1f3c1d5c0ca4 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:06:07.881674 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:06:07.929291 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac58ed-4a2b-467e-a3de-1f3c1d5c0ca4 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:06:07.960960 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c260cdeb-652e-436d-8022-1d1fc26cbcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:06:07.961594 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c260cdeb-652e-436d-8022-1d1fc26cbcbe None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:07.961874 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c260cdeb-652e-436d-8022-1d1fc26cbcbe None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:07.962145 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c260cdeb-652e-436d-8022-1d1fc26cbcbe None admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:07.962917 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c260cdeb-652e-436d-8022-1d1fc26cbcbe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:06:07.963220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c260cdeb-652e-436d-8022-1d1fc26cbcbe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:06:07.964686 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c260cdeb-652e-436d-8022-1d1fc26cbcbe None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:06:07.976004 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 654/1286] 10.222.0.41 () {56 vars in 1148 bytes} [Fri Jun 3 16:06:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:06:07.982324 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d3aba-cb36-4d46-af2a-e8d82f0f461a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:06:08.078487 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d3aba-cb36-4d46-af2a-e8d82f0f461a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:06:08.110510 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b2226f-f637-43ec-856f-51a49ce852f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:06:08.111133 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b2226f-f637-43ec-856f-51a49ce852f7 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:08.111448 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b2226f-f637-43ec-856f-51a49ce852f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:08.111723 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b2226f-f637-43ec-856f-51a49ce852f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:08.112534 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b2226f-f637-43ec-856f-51a49ce852f7 None admin] RBAC: Adding query filter params (role.id=75936a608b8149099ff54e317f3a6740, scope.project.id=c274254e510a468e9acbecc6298301cc, user.id=be901786f8934d559e0bac5d622f97cb) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:06:08.112840 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b2226f-f637-43ec-856f-51a49ce852f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:06:08.114244 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b2226f-f637-43ec-856f-51a49ce852f7 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:06:08.119853 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 633/1287] 10.222.0.41 () {56 vars in 1374 bytes} [Fri Jun 3 16:06:07 2022] GET /identity/v3/role_assignments?user.id=be901786f8934d559e0bac5d622f97cb&scope.project.id=c274254e510a468e9acbecc6298301cc&role.id=75936a608b8149099ff54e317f3a6740 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:06:10.348236 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533b69f-0598-4676-ae1d-a15652c178cc None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:10.348759 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533b69f-0598-4676-ae1d-a15652c178cc None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:10.348871 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533b69f-0598-4676-ae1d-a15652c178cc None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:10.350148 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 655/1288] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:06:10.356144 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10475b37-ad50-4bc5-ac94-50a720ec3598 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:10.356538 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10475b37-ad50-4bc5-ac94-50a720ec3598 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:10.356702 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10475b37-ad50-4bc5-ac94-50a720ec3598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:10.373008 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-10475b37-ad50-4bc5-ac94-50a720ec3598 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:06:10.374179 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-10475b37-ad50-4bc5-ac94-50a720ec3598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:06:10.402642 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:06:10.438465 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 634/1289] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:06:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:06:10.460051 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533b69f-0598-4676-ae1d-a15652c178cc None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:06:10.558294 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533b69f-0598-4676-ae1d-a15652c178cc None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:06:10.568625 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:06:10.591803 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77839397-7047-4f6a-9f24-a5ff8485ea04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:06:10.592588 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77839397-7047-4f6a-9f24-a5ff8485ea04 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:10.592871 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77839397-7047-4f6a-9f24-a5ff8485ea04 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:10.593144 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77839397-7047-4f6a-9f24-a5ff8485ea04 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:10.593783 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77839397-7047-4f6a-9f24-a5ff8485ea04 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:06:10.595134 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77839397-7047-4f6a-9f24-a5ff8485ea04 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:06:10.607432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-77839397-7047-4f6a-9f24-a5ff8485ea04 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 03 16:06:10.607432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:06:10.607432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 16:06:10.607432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 16:06:10.607432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 16:06:10.607432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 16:06:10.607432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 16:06:10.607432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:06:10.607432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 03 16:06:10.607432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:06:10.607432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 16:06:10.607432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:06:10.607432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:06:10.607432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:06:10.607432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:06:10.607432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:06:10.607432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:06:10.608648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:06:10.608648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:06:10.608648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:06:10.608648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:06:10.608648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:06:10.608648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:06:10.608648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 16:06:10.608648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 16:06:10.608648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 16:06:10.608648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 16:06:10.608648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:06:10.608648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:06:10.608648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:06:10.608648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:06:10.608648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:06:10.608648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:06:10.609912 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:06:10.609912 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:06:10.609912 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:06:10.609912 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:06:10.609912 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:06:10.609912 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:06:10.609912 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:06:10.609912 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:06:10.609912 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:06:10.609912 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:06:10.609912 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 16:06:10.609912 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:06:10.609912 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 03 16:06:10.609912 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:06:10.609912 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 656/1290] 10.222.0.41 () {56 vars in 1117 bytes} [Fri Jun 3 16:06:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:06:10.614599 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10475b37-ad50-4bc5-ac94-50a720ec3598 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:06:10.709468 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10475b37-ad50-4bc5-ac94-50a720ec3598 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:06:10.741916 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562e152-4aac-43bb-bcc4-8c4bccff5f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:06:10.742558 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562e152-4aac-43bb-bcc4-8c4bccff5f1c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:10.742843 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562e152-4aac-43bb-bcc4-8c4bccff5f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:10.743119 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562e152-4aac-43bb-bcc4-8c4bccff5f1c None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:10.743855 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d562e152-4aac-43bb-bcc4-8c4bccff5f1c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:06:10.744166 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d562e152-4aac-43bb-bcc4-8c4bccff5f1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:06:10.745555 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d562e152-4aac-43bb-bcc4-8c4bccff5f1c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:06:10.750190 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 635/1291] 10.222.0.41 () {56 vars in 1116 bytes} [Fri Jun 3 16:06:10 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:06:10.755687 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77839397-7047-4f6a-9f24-a5ff8485ea04 None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:06:10.852069 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77839397-7047-4f6a-9f24-a5ff8485ea04 None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:06:10.889620 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ddcaa-6c31-4c55-b5dd-61cefec0f10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:06:10.890281 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ddcaa-6c31-4c55-b5dd-61cefec0f10f None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:10.890571 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ddcaa-6c31-4c55-b5dd-61cefec0f10f None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:10.890845 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ddcaa-6c31-4c55-b5dd-61cefec0f10f None admin] PATH_INFO: `/v3/services` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:10.891573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0ddcaa-6c31-4c55-b5dd-61cefec0f10f None admin] RBAC: Adding query filter params (type=placement) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:06:10.891880 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0ddcaa-6c31-4c55-b5dd-61cefec0f10f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:06:10.893226 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0ddcaa-6c31-4c55-b5dd-61cefec0f10f None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:06:10.897551 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 657/1292] 10.222.0.41 () {56 vars in 1116 bytes} [Fri Jun 3 16:06:10 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:06:13.126710 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41e884-121e-4ecc-b7e4-7268e402439b None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:13.127323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41e884-121e-4ecc-b7e4-7268e402439b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:13.127323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41e884-121e-4ecc-b7e4-7268e402439b None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:13.128657 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 636/1293] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:06:13.133853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c2b28-c160-41af-afd2-9ba388d35afe None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:13.134269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c2b28-c160-41af-afd2-9ba388d35afe None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:13.134417 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c2b28-c160-41af-afd2-9ba388d35afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:13.155868 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-989c2b28-c160-41af-afd2-9ba388d35afe None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:06:13.157946 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-989c2b28-c160-41af-afd2-9ba388d35afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:06:13.188071 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:06:13.225244 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 658/1294] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:06:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:06:13.249351 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba41e884-121e-4ecc-b7e4-7268e402439b None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:06:13.279990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:06:13.347570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba41e884-121e-4ecc-b7e4-7268e402439b None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:06:13.379984 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fbb7b6-04db-461c-a9f1-91eb3989a7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:06:13.380577 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbb7b6-04db-461c-a9f1-91eb3989a7bd None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:13.380861 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbb7b6-04db-461c-a9f1-91eb3989a7bd None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:13.381136 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbb7b6-04db-461c-a9f1-91eb3989a7bd None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:13.381849 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fbb7b6-04db-461c-a9f1-91eb3989a7bd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:06:13.383204 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fbb7b6-04db-461c-a9f1-91eb3989a7bd None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:06:13.395640 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 637/1295] 10.222.0.41 () {60 vars in 1141 bytes} [Fri Jun 3 16:06:13 2022] POST /identity/v3/services => generated 246 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:06:15.607621 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa067f78-b422-4ea6-97ee-7984b86b4c3e None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:15.608132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa067f78-b422-4ea6-97ee-7984b86b4c3e None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:15.608132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa067f78-b422-4ea6-97ee-7984b86b4c3e None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:15.609456 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 659/1296] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:06:15.614636 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b061ddc-2d8f-40ee-9036-29b9b6b486c3 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:15.615000 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b061ddc-2d8f-40ee-9036-29b9b6b486c3 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:15.615201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b061ddc-2d8f-40ee-9036-29b9b6b486c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:15.632023 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-4b061ddc-2d8f-40ee-9036-29b9b6b486c3 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:06:15.633189 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4b061ddc-2d8f-40ee-9036-29b9b6b486c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:06:15.702824 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 638/1297] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:06:15 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:06:15.724775 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa067f78-b422-4ea6-97ee-7984b86b4c3e None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:06:15.822354 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa067f78-b422-4ea6-97ee-7984b86b4c3e None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:06:15.855371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f24b4-8a04-4be1-ab12-331673b4924b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:06:15.856097 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f24b4-8a04-4be1-ab12-331673b4924b None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:15.856380 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f24b4-8a04-4be1-ab12-331673b4924b None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:15.856654 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f24b4-8a04-4be1-ab12-331673b4924b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:15.857296 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f24b4-8a04-4be1-ab12-331673b4924b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:06:15.858701 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f24b4-8a04-4be1-ab12-331673b4924b None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:06:15.870961 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-ab3f24b4-8a04-4be1-ab12-331673b4924b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 03 16:06:15.870961 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:06:15.870961 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 16:06:15.870961 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 16:06:15.870961 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 16:06:15.870961 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 16:06:15.870961 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 16:06:15.870961 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:06:15.870961 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 03 16:06:15.870961 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:06:15.870961 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 16:06:15.870961 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:06:15.870961 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:06:15.870961 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:06:15.870961 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:06:15.870961 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:06:15.870961 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:06:15.872264 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:06:15.872264 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:06:15.872264 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:06:15.872264 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:06:15.872264 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:06:15.872264 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:06:15.872264 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 16:06:15.872264 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 16:06:15.872264 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 16:06:15.872264 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 16:06:15.872264 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:06:15.872264 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:06:15.872264 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:06:15.872264 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:06:15.872264 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:06:15.872264 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:06:15.873198 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:06:15.873198 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:06:15.873198 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:06:15.873198 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:06:15.873198 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:06:15.873198 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:06:15.873198 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:06:15.873198 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:06:15.873198 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:06:15.873198 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:06:15.873198 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 16:06:15.873198 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:06:15.873198 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 03 16:06:15.873198 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:06:15.873198 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 660/1298] 10.222.0.41 () {56 vars in 1117 bytes} [Fri Jun 3 16:06:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:06:15.877928 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b061ddc-2d8f-40ee-9036-29b9b6b486c3 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:06:15.895308 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:06:15.971916 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b061ddc-2d8f-40ee-9036-29b9b6b486c3 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:06:16.005458 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d431e77c-56b7-4810-b8c7-ced95c0d5686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:06:16.006305 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d431e77c-56b7-4810-b8c7-ced95c0d5686 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:16.006657 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d431e77c-56b7-4810-b8c7-ced95c0d5686 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:16.006973 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d431e77c-56b7-4810-b8c7-ced95c0d5686 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:16.007865 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d431e77c-56b7-4810-b8c7-ced95c0d5686 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:06:16.008280 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d431e77c-56b7-4810-b8c7-ced95c0d5686 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:06:16.009832 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d431e77c-56b7-4810-b8c7-ced95c0d5686 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:06:16.015890 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 639/1299] 10.222.0.41 () {56 vars in 1116 bytes} [Fri Jun 3 16:06:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:06:16.022192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f24b4-8a04-4be1-ab12-331673b4924b None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:06:16.054229 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:06:16.122323 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f24b4-8a04-4be1-ab12-331673b4924b None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:06:16.154899 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a9e10-62f5-4814-8368-29db25018f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:06:16.155575 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a9e10-62f5-4814-8368-29db25018f71 None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:16.155864 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a9e10-62f5-4814-8368-29db25018f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:16.156144 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a9e10-62f5-4814-8368-29db25018f71 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:16.156929 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85a9e10-62f5-4814-8368-29db25018f71 None admin] RBAC: Adding query filter params (interface=public, service_id=d09c307028914f2c9b449dcbb8ab5f37, region_id=RegionOne) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:06:16.157245 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85a9e10-62f5-4814-8368-29db25018f71 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:06:16.158599 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85a9e10-62f5-4814-8368-29db25018f71 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:06:16.163556 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 661/1300] 10.222.0.41 () {56 vars in 1251 bytes} [Fri Jun 3 16:06:16 2022] GET /identity/v3/endpoints?service_id=d09c307028914f2c9b449dcbb8ab5f37&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:06:16.169543 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d431e77c-56b7-4810-b8c7-ced95c0d5686 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:06:16.248237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:06:16.265960 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d431e77c-56b7-4810-b8c7-ced95c0d5686 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:06:16.299021 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2db2f-d004-46aa-8dc8-022ef4d97356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:06:16.299693 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2db2f-d004-46aa-8dc8-022ef4d97356 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:16.299977 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2db2f-d004-46aa-8dc8-022ef4d97356 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:16.300251 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2db2f-d004-46aa-8dc8-022ef4d97356 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:16.300923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b2db2f-d004-46aa-8dc8-022ef4d97356 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:06:16.302278 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b2db2f-d004-46aa-8dc8-022ef4d97356 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:06:16.308905 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 640/1301] 10.222.0.41 () {56 vars in 1087 bytes} [Fri Jun 3 16:06:16 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:06:18.528988 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5262238-1cbb-4f8e-bfc2-11c668682c11 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:18.529490 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5262238-1cbb-4f8e-bfc2-11c668682c11 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:18.529490 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5262238-1cbb-4f8e-bfc2-11c668682c11 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:18.530840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 662/1302] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:06:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:06:18.535964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b3de7-4217-451b-af26-c465982225ba None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:18.536349 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b3de7-4217-451b-af26-c465982225ba None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:18.536515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b3de7-4217-451b-af26-c465982225ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:18.553468 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-391b3de7-4217-451b-af26-c465982225ba None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:06:18.554633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-391b3de7-4217-451b-af26-c465982225ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:06:18.617072 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 641/1303] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:06:18 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 16:06:18.639682 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5262238-1cbb-4f8e-bfc2-11c668682c11 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:06:18.731432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:06:18.738562 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5262238-1cbb-4f8e-bfc2-11c668682c11 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:06:18.771824 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f36f82-235b-4104-b691-152db119ea9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:06:18.772607 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f36f82-235b-4104-b691-152db119ea9f None admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:18.772893 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f36f82-235b-4104-b691-152db119ea9f None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:18.773424 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f36f82-235b-4104-b691-152db119ea9f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:18.774072 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f36f82-235b-4104-b691-152db119ea9f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:06:18.775458 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f36f82-235b-4104-b691-152db119ea9f None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:06:18.787238 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application [None req-90f36f82-235b-4104-b691-152db119ea9f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 03 16:06:18.787238 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:06:18.787238 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 16:06:18.787238 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 16:06:18.787238 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 16:06:18.787238 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 16:06:18.787238 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 16:06:18.787238 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:06:18.787238 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 03 16:06:18.787238 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:06:18.787238 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 16:06:18.787238 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:06:18.787238 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 16:06:18.787238 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 16:06:18.787238 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 16:06:18.787238 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 16:06:18.787238 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 16:06:18.788243 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 16:06:18.788243 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 16:06:18.788243 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 16:06:18.788243 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 16:06:18.788243 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 16:06:18.788243 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 16:06:18.788243 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 16:06:18.788243 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 16:06:18.788243 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 16:06:18.788243 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 16:06:18.788243 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 16:06:18.788243 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 16:06:18.788243 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 16:06:18.788243 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 16:06:18.788243 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 16:06:18.788243 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 16:06:18.789265 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 16:06:18.789265 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application with Lock( Jun 03 16:06:18.789265 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 16:06:18.789265 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self._enter() Jun 03 16:06:18.789265 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 16:06:18.789265 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 16:06:18.789265 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 16:06:18.789265 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application return self.creator() Jun 03 16:06:18.789265 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 16:06:18.789265 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application created_value = creator( Jun 03 16:06:18.789265 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 16:06:18.789265 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 16:06:18.789265 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 03 16:06:18.789265 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: ERROR keystone.server.flask.application Jun 03 16:06:18.789265 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 663/1304] 10.222.0.41 () {56 vars in 1117 bytes} [Fri Jun 3 16:06:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 16:06:18.794353 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391b3de7-4217-451b-af26-c465982225ba None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:06:18.866647 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:06:18.891579 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391b3de7-4217-451b-af26-c465982225ba None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:06:18.923984 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3513d9e-0eaf-4622-8f35-7029004469d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:06:18.924624 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3513d9e-0eaf-4622-8f35-7029004469d0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:18.924906 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3513d9e-0eaf-4622-8f35-7029004469d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:18.925178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3513d9e-0eaf-4622-8f35-7029004469d0 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:18.925871 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3513d9e-0eaf-4622-8f35-7029004469d0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:06:18.926174 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3513d9e-0eaf-4622-8f35-7029004469d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:06:18.927550 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3513d9e-0eaf-4622-8f35-7029004469d0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:06:18.932494 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 642/1305] 10.222.0.41 () {56 vars in 1116 bytes} [Fri Jun 3 16:06:18 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:06:18.938340 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f36f82-235b-4104-b691-152db119ea9f None admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:06:19.035585 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f36f82-235b-4104-b691-152db119ea9f None admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:06:19.067526 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae36f7-f69e-462c-9d23-5e45afd87554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:06:19.068107 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae36f7-f69e-462c-9d23-5e45afd87554 None admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:06:19.068394 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae36f7-f69e-462c-9d23-5e45afd87554 None admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:06:19.068676 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae36f7-f69e-462c-9d23-5e45afd87554 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:06:19.069402 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ae36f7-f69e-462c-9d23-5e45afd87554 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:06:19.070746 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ae36f7-f69e-462c-9d23-5e45afd87554 None admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:06:19.089432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 664/1306] 10.222.0.41 () {60 vars in 1144 bytes} [Fri Jun 3 16:06:18 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:06:28.929971 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=30 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:06:29.084386 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:07:22.209094 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea672c9-788d-41bd-8e8e-e579b481cc6b None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:22.210753 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea672c9-788d-41bd-8e8e-e579b481cc6b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:22.211258 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea672c9-788d-41bd-8e8e-e579b481cc6b None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:22.216785 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 643/1307] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:07:22 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:07:22.218172 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a2af6-f208-4e89-8596-18f288bd35e2 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:22.218588 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a2af6-f208-4e89-8596-18f288bd35e2 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:22.218796 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a2af6-f208-4e89-8596-18f288bd35e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:22.364431 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-264a2af6-f208-4e89-8596-18f288bd35e2 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:07:22.365702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-264a2af6-f208-4e89-8596-18f288bd35e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:07:22.769962 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 665/1308] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:07:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 553 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:07:22.824873 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff18e1-58d6-4997-b30d-360e484b945c None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:22.825157 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff18e1-58d6-4997-b30d-360e484b945c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:22.825677 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff18e1-58d6-4997-b30d-360e484b945c None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:22.827534 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 644/1309] 10.222.0.41 () {54 vars in 964 bytes} [Fri Jun 3 16:07:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:07:22.835170 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570955be-783d-465e-9cbd-1e416de9c373 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:22.835824 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570955be-783d-465e-9cbd-1e416de9c373 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:22.836339 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570955be-783d-465e-9cbd-1e416de9c373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:22.918678 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-570955be-783d-465e-9cbd-1e416de9c373 None None] MFA Rules not processed for user `aeeaad0680074179b3d29abcf0d51515`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:07:22.920299 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-570955be-783d-465e-9cbd-1e416de9c373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:07:22.942401 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:07:23.011626 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff18e1-58d6-4997-b30d-360e484b945c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:07:23.013826 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e5ff18e1-58d6-4997-b30d-360e484b945c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:07:23.018496 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 666/1310] 10.222.0.41 () {58 vars in 1061 bytes} [Fri Jun 3 16:07:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:07:23.150687 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff18e1-58d6-4997-b30d-360e484b945c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:07:23.239521 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d22d3f-6ed6-42ed-a1ac-917085418f14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:07:23.241156 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d22d3f-6ed6-42ed-a1ac-917085418f14 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:23.241628 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d22d3f-6ed6-42ed-a1ac-917085418f14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:23.242109 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d22d3f-6ed6-42ed-a1ac-917085418f14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:23.244155 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-99d22d3f-6ed6-42ed-a1ac-917085418f14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:07:23.309749 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:07:23.371193 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d22d3f-6ed6-42ed-a1ac-917085418f14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:07:23.373109 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d22d3f-6ed6-42ed-a1ac-917085418f14 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:07:23.393616 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 645/1311] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:07:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 16:07:25.990543 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70680d3e-485e-4161-932c-b9ec47613db8 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:25.991572 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70680d3e-485e-4161-932c-b9ec47613db8 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:25.991693 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70680d3e-485e-4161-932c-b9ec47613db8 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:25.992986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 667/1312] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:07:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:07:25.998376 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a361e-4326-4b1c-a5bb-b3da5f9c660e None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:25.998792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a361e-4326-4b1c-a5bb-b3da5f9c660e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:25.998932 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a361e-4326-4b1c-a5bb-b3da5f9c660e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:26.094299 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-082a361e-4326-4b1c-a5bb-b3da5f9c660e None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:07:26.095585 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-082a361e-4326-4b1c-a5bb-b3da5f9c660e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:07:26.198246 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 646/1313] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:07:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:07:26.236523 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70680d3e-485e-4161-932c-b9ec47613db8 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:07:26.238139 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-70680d3e-485e-4161-932c-b9ec47613db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:07:26.362585 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70680d3e-485e-4161-932c-b9ec47613db8 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:07:26.434648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:07:26.437404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23eccc-0650-4752-bbd4-f8d30c6c32d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:07:26.438233 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23eccc-0650-4752-bbd4-f8d30c6c32d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:26.438671 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23eccc-0650-4752-bbd4-f8d30c6c32d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:26.439116 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23eccc-0650-4752-bbd4-f8d30c6c32d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:26.440787 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-6d23eccc-0650-4752-bbd4-f8d30c6c32d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:07:26.548755 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23eccc-0650-4752-bbd4-f8d30c6c32d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:07:26.550071 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23eccc-0650-4752-bbd4-f8d30c6c32d6 service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:07:26.568856 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 668/1314] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:07:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:07:29.104947 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7049f064-b7d0-4450-b6a8-a9e5c74422be None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:29.105451 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7049f064-b7d0-4450-b6a8-a9e5c74422be None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:29.105451 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7049f064-b7d0-4450-b6a8-a9e5c74422be None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:29.106791 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 647/1315] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:07:29.112082 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6db521-817b-4f82-877e-97ca49557b57 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:29.112458 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6db521-817b-4f82-877e-97ca49557b57 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:29.112637 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6db521-817b-4f82-877e-97ca49557b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:29.174458 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-cc6db521-817b-4f82-877e-97ca49557b57 None None] MFA Rules not processed for user `f6a0ad4d1a084c0db8eb52bd4ed3a69a`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:07:29.175672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-cc6db521-817b-4f82-877e-97ca49557b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:07:29.226413 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:07:29.271191 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 669/1316] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:07:29 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:07:29.310539 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7049f064-b7d0-4450-b6a8-a9e5c74422be None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:07:29.312113 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7049f064-b7d0-4450-b6a8-a9e5c74422be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:07:29.322317 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:07:29.414043 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7049f064-b7d0-4450-b6a8-a9e5c74422be None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:07:29.484907 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87268a98-6bab-4c5c-a760-e2618890552d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:07:29.485539 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87268a98-6bab-4c5c-a760-e2618890552d service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:29.485825 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87268a98-6bab-4c5c-a760-e2618890552d service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:29.486103 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87268a98-6bab-4c5c-a760-e2618890552d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:29.487761 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-87268a98-6bab-4c5c-a760-e2618890552d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:07:29.548249 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:07:29.593404 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87268a98-6bab-4c5c-a760-e2618890552d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:07:29.594846 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87268a98-6bab-4c5c-a760-e2618890552d service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:07:29.614423 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 648/1317] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:07:29 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:07:32.389827 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13494c0-4320-4472-8b3c-662642197894 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:32.390364 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13494c0-4320-4472-8b3c-662642197894 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:32.390364 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13494c0-4320-4472-8b3c-662642197894 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:32.391817 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 670/1318] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:07:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:07:32.397229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a64001-3a7e-4700-8e0e-f15c2e064ca6 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:32.397644 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a64001-3a7e-4700-8e0e-f15c2e064ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:32.397774 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a64001-3a7e-4700-8e0e-f15c2e064ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:32.460111 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-f9a64001-3a7e-4700-8e0e-f15c2e064ca6 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:07:32.461299 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f9a64001-3a7e-4700-8e0e-f15c2e064ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:07:32.553155 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 649/1319] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:07:32 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:07:32.606095 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccbc4a-daef-439c-8d57-dd3eaadce4e1 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:32.606382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccbc4a-daef-439c-8d57-dd3eaadce4e1 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:32.606664 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccbc4a-daef-439c-8d57-dd3eaadce4e1 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:32.608077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 671/1320] 10.222.0.41 () {54 vars in 964 bytes} [Fri Jun 3 16:07:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:07:32.613884 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a76ae-5d92-4e18-8b0f-7299345b548d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:32.614175 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a76ae-5d92-4e18-8b0f-7299345b548d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:32.614453 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a76ae-5d92-4e18-8b0f-7299345b548d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:32.634934 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:07:32.687166 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-371a76ae-5d92-4e18-8b0f-7299345b548d None None] MFA Rules not processed for user `aeeaad0680074179b3d29abcf0d51515`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:07:32.688401 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-371a76ae-5d92-4e18-8b0f-7299345b548d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:07:32.761992 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 650/1321] 10.222.0.41 () {58 vars in 1061 bytes} [Fri Jun 3 16:07:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:07:32.771260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ccbc4a-daef-439c-8d57-dd3eaadce4e1 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:07:32.772821 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-69ccbc4a-daef-439c-8d57-dd3eaadce4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:07:32.872383 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ccbc4a-daef-439c-8d57-dd3eaadce4e1 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:07:32.942803 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb10e5e-8fb4-4b5d-a451-3324959f7a82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:07:32.943463 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb10e5e-8fb4-4b5d-a451-3324959f7a82 service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:32.943745 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb10e5e-8fb4-4b5d-a451-3324959f7a82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:32.944017 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb10e5e-8fb4-4b5d-a451-3324959f7a82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:32.945617 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-feb10e5e-8fb4-4b5d-a451-3324959f7a82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:07:32.952623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:07:33.051367 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb10e5e-8fb4-4b5d-a451-3324959f7a82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:07:33.053295 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb10e5e-8fb4-4b5d-a451-3324959f7a82 service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:07:33.071189 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 672/1322] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:07:32 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:07:36.417891 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc22e7-6ad7-4c51-9925-f47548a1b5f4 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:36.418408 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc22e7-6ad7-4c51-9925-f47548a1b5f4 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:36.418527 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc22e7-6ad7-4c51-9925-f47548a1b5f4 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:36.419845 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 651/1323] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:07:36.425084 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf246fd-6e03-46ca-a33e-fd69e3b82f25 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:36.425488 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf246fd-6e03-46ca-a33e-fd69e3b82f25 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:36.425656 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf246fd-6e03-46ca-a33e-fd69e3b82f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:36.490029 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-dbf246fd-6e03-46ca-a33e-fd69e3b82f25 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:07:36.491216 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-dbf246fd-6e03-46ca-a33e-fd69e3b82f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:07:36.546772 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:07:36.585912 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 673/1324] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:07:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:07:36.622264 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc22e7-6ad7-4c51-9925-f47548a1b5f4 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:07:36.623869 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e8dc22e7-6ad7-4c51-9925-f47548a1b5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:07:36.726089 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc22e7-6ad7-4c51-9925-f47548a1b5f4 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:07:36.738557 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:07:36.799187 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7147e3e-f1fd-4755-bccf-7098c0d2db08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:07:36.799776 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7147e3e-f1fd-4755-bccf-7098c0d2db08 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:36.800058 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7147e3e-f1fd-4755-bccf-7098c0d2db08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:36.800332 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7147e3e-f1fd-4755-bccf-7098c0d2db08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:36.801968 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c7147e3e-f1fd-4755-bccf-7098c0d2db08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:07:36.913196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7147e3e-f1fd-4755-bccf-7098c0d2db08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:07:36.914639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7147e3e-f1fd-4755-bccf-7098c0d2db08 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:07:36.933566 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 652/1325] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:07:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:07:39.224313 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5516c-0455-44e3-8ae9-83ecb95dae7e None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:39.224835 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5516c-0455-44e3-8ae9-83ecb95dae7e None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:39.224919 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5516c-0455-44e3-8ae9-83ecb95dae7e None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:39.226389 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 674/1326] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:07:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:07:39.231885 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a67b5-d98a-4d0b-b8bd-4b0b14a9ef38 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:39.232332 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a67b5-d98a-4d0b-b8bd-4b0b14a9ef38 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:39.232594 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a67b5-d98a-4d0b-b8bd-4b0b14a9ef38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:39.281815 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:07:39.299917 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-572a67b5-d98a-4d0b-b8bd-4b0b14a9ef38 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:07:39.301165 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-572a67b5-d98a-4d0b-b8bd-4b0b14a9ef38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:07:39.397886 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 653/1327] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:07:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:07:39.433628 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5516c-0455-44e3-8ae9-83ecb95dae7e None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:07:39.435145 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-bdd5516c-0455-44e3-8ae9-83ecb95dae7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:07:39.545377 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5516c-0455-44e3-8ae9-83ecb95dae7e None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:07:39.617092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062076ef-d6c9-4eb0-94dc-25fe8f4685c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:07:39.617732 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062076ef-d6c9-4eb0-94dc-25fe8f4685c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:39.618046 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062076ef-d6c9-4eb0-94dc-25fe8f4685c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:39.618321 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062076ef-d6c9-4eb0-94dc-25fe8f4685c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:39.619977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-062076ef-d6c9-4eb0-94dc-25fe8f4685c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:07:39.636730 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:07:39.728400 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062076ef-d6c9-4eb0-94dc-25fe8f4685c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:07:39.729832 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062076ef-d6c9-4eb0-94dc-25fe8f4685c0 service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:07:39.752043 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 675/1328] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:07:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:07:42.072092 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf6657-109b-45e1-aba3-c75ac314fc6f None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:42.072649 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf6657-109b-45e1-aba3-c75ac314fc6f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:42.072649 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf6657-109b-45e1-aba3-c75ac314fc6f None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:42.073983 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 654/1329] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:07:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:07:42.079206 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87882ab-41a2-4459-bcc2-c11c00b59698 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:42.079648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87882ab-41a2-4459-bcc2-c11c00b59698 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:42.079928 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87882ab-41a2-4459-bcc2-c11c00b59698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:42.146338 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-d87882ab-41a2-4459-bcc2-c11c00b59698 None None] MFA Rules not processed for user `f6a0ad4d1a084c0db8eb52bd4ed3a69a`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:07:42.147558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-d87882ab-41a2-4459-bcc2-c11c00b59698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:07:42.227338 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:07:42.263764 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 676/1330] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:07:42 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:07:42.301602 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbf6657-109b-45e1-aba3-c75ac314fc6f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:07:42.303762 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8dbf6657-109b-45e1-aba3-c75ac314fc6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:07:42.405514 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbf6657-109b-45e1-aba3-c75ac314fc6f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:07:42.425441 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:07:42.479397 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d3a87-8a9c-4718-a9be-22d56c280edf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:07:42.480048 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d3a87-8a9c-4718-a9be-22d56c280edf service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:42.480334 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d3a87-8a9c-4718-a9be-22d56c280edf service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:42.480610 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d3a87-8a9c-4718-a9be-22d56c280edf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:42.482241 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ff0d3a87-8a9c-4718-a9be-22d56c280edf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:07:42.589099 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d3a87-8a9c-4718-a9be-22d56c280edf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:07:42.590579 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d3a87-8a9c-4718-a9be-22d56c280edf service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:07:42.738075 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 655/1331] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:07:42 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:07:45.228224 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac5a0f1-b0e6-4665-a536-4eb39ebd2214 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:45.228790 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac5a0f1-b0e6-4665-a536-4eb39ebd2214 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:45.228790 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac5a0f1-b0e6-4665-a536-4eb39ebd2214 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:45.230212 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 677/1332] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:07:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:07:45.236230 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4261b-202c-4556-91c8-cdfcfe58ad01 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:45.236747 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4261b-202c-4556-91c8-cdfcfe58ad01 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:45.236945 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4261b-202c-4556-91c8-cdfcfe58ad01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:45.301911 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:07:45.304710 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-76d4261b-202c-4556-91c8-cdfcfe58ad01 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:07:45.306518 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-76d4261b-202c-4556-91c8-cdfcfe58ad01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:07:45.403321 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 656/1333] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:07:45.438882 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac5a0f1-b0e6-4665-a536-4eb39ebd2214 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:07:45.440455 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-7ac5a0f1-b0e6-4665-a536-4eb39ebd2214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:07:45.542715 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac5a0f1-b0e6-4665-a536-4eb39ebd2214 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:07:45.615731 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bfcd30-dbca-49ca-90ef-e0933d0fcb4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:07:45.616301 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfcd30-dbca-49ca-90ef-e0933d0fcb4f service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:45.616589 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfcd30-dbca-49ca-90ef-e0933d0fcb4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:45.616868 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfcd30-dbca-49ca-90ef-e0933d0fcb4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:45.618510 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-49bfcd30-dbca-49ca-90ef-e0933d0fcb4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:07:45.647520 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:07:45.736238 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bfcd30-dbca-49ca-90ef-e0933d0fcb4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:07:45.737671 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bfcd30-dbca-49ca-90ef-e0933d0fcb4f service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:07:45.757271 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 678/1334] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:07:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:07:48.044915 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08615205-b879-4c86-af6f-7c9e8159d3e4 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:48.045437 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08615205-b879-4c86-af6f-7c9e8159d3e4 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:48.045562 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08615205-b879-4c86-af6f-7c9e8159d3e4 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:48.046872 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 657/1335] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:07:48.052069 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c838b7-90a5-45ea-9780-5aa60e557b29 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:48.052465 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c838b7-90a5-45ea-9780-5aa60e557b29 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:48.052625 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c838b7-90a5-45ea-9780-5aa60e557b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:48.116028 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-c1c838b7-90a5-45ea-9780-5aa60e557b29 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:07:48.117179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-c1c838b7-90a5-45ea-9780-5aa60e557b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:07:48.189959 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:07:48.212871 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 679/1336] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:07:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:07:48.255491 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08615205-b879-4c86-af6f-7c9e8159d3e4 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:07:48.258359 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-08615205-b879-4c86-af6f-7c9e8159d3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:07:48.361251 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08615205-b879-4c86-af6f-7c9e8159d3e4 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:07:48.388818 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:07:48.433155 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7807c7-e3ff-4cbc-aeb9-4b1a26af4db2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:07:48.433798 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7807c7-e3ff-4cbc-aeb9-4b1a26af4db2 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:48.434082 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7807c7-e3ff-4cbc-aeb9-4b1a26af4db2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:48.434361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7807c7-e3ff-4cbc-aeb9-4b1a26af4db2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:48.436038 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3c7807c7-e3ff-4cbc-aeb9-4b1a26af4db2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:07:48.543739 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7807c7-e3ff-4cbc-aeb9-4b1a26af4db2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:07:48.545162 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7807c7-e3ff-4cbc-aeb9-4b1a26af4db2 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:07:48.562826 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 658/1337] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:07:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:07:51.253775 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4304262-fb84-4d84-b7c5-ce1c02990b80 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:51.254166 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4304262-fb84-4d84-b7c5-ce1c02990b80 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:51.254335 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4304262-fb84-4d84-b7c5-ce1c02990b80 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:51.255793 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 680/1338] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:07:51.261030 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07465af0-498a-4827-a7e0-f39497f3b801 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:51.261435 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07465af0-498a-4827-a7e0-f39497f3b801 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:51.261579 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07465af0-498a-4827-a7e0-f39497f3b801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:51.324963 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-07465af0-498a-4827-a7e0-f39497f3b801 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:07:51.326159 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-07465af0-498a-4827-a7e0-f39497f3b801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:07:51.334239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:07:51.419866 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 659/1339] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:07:51 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:07:51.455664 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4304262-fb84-4d84-b7c5-ce1c02990b80 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:07:51.457220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-c4304262-fb84-4d84-b7c5-ce1c02990b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:07:51.576273 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4304262-fb84-4d84-b7c5-ce1c02990b80 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:07:51.647860 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cbc48d-53be-4a42-8e1e-9616e150c629 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:07:51.648499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbc48d-53be-4a42-8e1e-9616e150c629 service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:51.648779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbc48d-53be-4a42-8e1e-9616e150c629 service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:51.649053 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbc48d-53be-4a42-8e1e-9616e150c629 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:51.650682 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-96cbc48d-53be-4a42-8e1e-9616e150c629 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:07:51.685509 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:07:51.757163 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cbc48d-53be-4a42-8e1e-9616e150c629 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:07:51.758593 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cbc48d-53be-4a42-8e1e-9616e150c629 service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:07:51.776781 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 681/1340] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:07:51 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:07:55.997689 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18b162-7407-4876-853d-732843ba536e None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:55.998084 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18b162-7407-4876-853d-732843ba536e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:55.998247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18b162-7407-4876-853d-732843ba536e None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:55.999675 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 660/1341] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:07:56.004985 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7227700-28d7-48ce-886e-dd79bbabd3e1 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:56.005397 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7227700-28d7-48ce-886e-dd79bbabd3e1 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:56.005545 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7227700-28d7-48ce-886e-dd79bbabd3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:56.078147 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-e7227700-28d7-48ce-886e-dd79bbabd3e1 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:07:56.079527 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-e7227700-28d7-48ce-886e-dd79bbabd3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:07:56.174865 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:07:56.188150 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 682/1342] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:07:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:07:56.227403 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f18b162-7407-4876-853d-732843ba536e None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:07:56.229595 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6f18b162-7407-4876-853d-732843ba536e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:07:56.329299 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f18b162-7407-4876-853d-732843ba536e None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:07:56.333323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79deda0-9a44-4800-9321-29fc260c1c94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:07:56.333963 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79deda0-9a44-4800-9321-29fc260c1c94 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:56.334243 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79deda0-9a44-4800-9321-29fc260c1c94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:56.334514 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79deda0-9a44-4800-9321-29fc260c1c94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:56.343476 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79deda0-9a44-4800-9321-29fc260c1c94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:07:56.345084 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79deda0-9a44-4800-9321-29fc260c1c94 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:07:56.355053 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 661/1343] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:07:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:07:59.153177 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a486b-b381-4c84-b388-740f8141abcc None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:59.153715 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a486b-b381-4c84-b388-740f8141abcc None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:59.153715 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a486b-b381-4c84-b388-740f8141abcc None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:59.155274 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 683/1344] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:07:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:07:59.160306 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e26cbd-6073-4457-98b5-6d2b945b5bc3 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:59.160743 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e26cbd-6073-4457-98b5-6d2b945b5bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:59.160902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e26cbd-6073-4457-98b5-6d2b945b5bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:59.188490 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-a0e26cbd-6073-4457-98b5-6d2b945b5bc3 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:07:59.189784 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a0e26cbd-6073-4457-98b5-6d2b945b5bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:07:59.216893 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:07:59.272793 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 662/1345] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:07:59 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:07:59.309877 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20a486b-b381-4c84-b388-740f8141abcc None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:07:59.320588 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20a486b-b381-4c84-b388-740f8141abcc None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:07:59.324254 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf445152-d096-4a0e-bb1a-7fb85ca7d95b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:07:59.324892 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf445152-d096-4a0e-bb1a-7fb85ca7d95b service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:07:59.325177 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf445152-d096-4a0e-bb1a-7fb85ca7d95b service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:07:59.325456 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf445152-d096-4a0e-bb1a-7fb85ca7d95b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:07:59.334624 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf445152-d096-4a0e-bb1a-7fb85ca7d95b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:07:59.336067 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf445152-d096-4a0e-bb1a-7fb85ca7d95b service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:07:59.345489 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 684/1346] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:07:59 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:08:03.575515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aaa1a1-5576-4327-8fcc-a299d53a50f5 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:03.576090 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aaa1a1-5576-4327-8fcc-a299d53a50f5 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:03.576090 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aaa1a1-5576-4327-8fcc-a299d53a50f5 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:03.577437 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 663/1347] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:08:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:08:03.582704 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c463af-2fd5-4da6-9080-ad40fe9fd77b None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:03.583100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c463af-2fd5-4da6-9080-ad40fe9fd77b None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:03.583239 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c463af-2fd5-4da6-9080-ad40fe9fd77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:03.602333 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-d3c463af-2fd5-4da6-9080-ad40fe9fd77b None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:08:03.603587 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-d3c463af-2fd5-4da6-9080-ad40fe9fd77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:08:03.611312 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 685/1348] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:08:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:08:03.649814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aaa1a1-5576-4327-8fcc-a299d53a50f5 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:08:03.660725 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aaa1a1-5576-4327-8fcc-a299d53a50f5 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:08:03.664315 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a872f5-36c3-4126-bcd4-9f572c2076f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:08:03.664950 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a872f5-36c3-4126-bcd4-9f572c2076f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:03.665240 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a872f5-36c3-4126-bcd4-9f572c2076f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:03.665521 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a872f5-36c3-4126-bcd4-9f572c2076f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:03.674436 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a872f5-36c3-4126-bcd4-9f572c2076f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:08:03.675919 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a872f5-36c3-4126-bcd4-9f572c2076f1 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:08:03.685780 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 664/1349] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:08:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:08:09.045309 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776eda6e-eca0-4015-a220-2a688b443a1f None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:09.045824 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776eda6e-eca0-4015-a220-2a688b443a1f None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:09.045824 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776eda6e-eca0-4015-a220-2a688b443a1f None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:09.047231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 686/1350] 10.222.0.41 () {54 vars in 921 bytes} [Fri Jun 3 16:08:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:08:09.060633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c18667-4303-4707-84f2-bb38a3c52cc4 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:09.061457 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c18667-4303-4707-84f2-bb38a3c52cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:09.061918 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c18667-4303-4707-84f2-bb38a3c52cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:09.167925 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-37c18667-4303-4707-84f2-bb38a3c52cc4 None None] MFA Rules not processed for user `be901786f8934d559e0bac5d622f97cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:08:09.169598 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-37c18667-4303-4707-84f2-bb38a3c52cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:08:09.263701 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 665/1351] 10.222.0.41 () {58 vars in 1018 bytes} [Fri Jun 3 16:08:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:08:13.597640 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=13 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:08:13.661297 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a8b8f-84a0-4c6f-b247-9ba734a32323 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:13.662011 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a8b8f-84a0-4c6f-b247-9ba734a32323 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:13.662294 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a8b8f-84a0-4c6f-b247-9ba734a32323 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:13.663868 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 687/1352] 10.222.0.41 () {54 vars in 921 bytes} [Fri Jun 3 16:08:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:08:13.668730 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45edcd5-4f6a-4242-816f-e0f587bd684e None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:13.669193 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45edcd5-4f6a-4242-816f-e0f587bd684e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:13.669376 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45edcd5-4f6a-4242-816f-e0f587bd684e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:13.689816 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-e45edcd5-4f6a-4242-816f-e0f587bd684e None None] MFA Rules not processed for user `be901786f8934d559e0bac5d622f97cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:08:13.692017 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e45edcd5-4f6a-4242-816f-e0f587bd684e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:08:13.701582 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 666/1353] 10.222.0.41 () {58 vars in 1018 bytes} [Fri Jun 3 16:08:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:08:15.269139 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538f21f-3153-4c6c-87c4-4bb06df9b8b3 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:15.269845 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538f21f-3153-4c6c-87c4-4bb06df9b8b3 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:15.269845 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538f21f-3153-4c6c-87c4-4bb06df9b8b3 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:15.271924 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 688/1354] 10.222.0.41 () {54 vars in 921 bytes} [Fri Jun 3 16:08:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:08:15.277664 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9642ff82-3f21-415f-9b1c-3370496576c9 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:15.278230 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9642ff82-3f21-415f-9b1c-3370496576c9 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:15.278230 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9642ff82-3f21-415f-9b1c-3370496576c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:15.297358 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-9642ff82-3f21-415f-9b1c-3370496576c9 None None] MFA Rules not processed for user `be901786f8934d559e0bac5d622f97cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:08:15.298561 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9642ff82-3f21-415f-9b1c-3370496576c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:08:15.307350 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 667/1355] 10.222.0.41 () {58 vars in 1018 bytes} [Fri Jun 3 16:08:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:08:19.102073 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a9ac0-d5f2-4a80-8f13-ab3dbdfb1b02 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:19.102561 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a9ac0-d5f2-4a80-8f13-ab3dbdfb1b02 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:19.102680 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a9ac0-d5f2-4a80-8f13-ab3dbdfb1b02 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:19.104175 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 689/1356] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:08:19.109139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d111d5-0bf0-4349-aa49-4fd6f55c2b06 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:19.109975 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d111d5-0bf0-4349-aa49-4fd6f55c2b06 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:19.110262 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d111d5-0bf0-4349-aa49-4fd6f55c2b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:19.128327 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-25d111d5-0bf0-4349-aa49-4fd6f55c2b06 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:08:19.129507 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-25d111d5-0bf0-4349-aa49-4fd6f55c2b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:08:19.137880 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 668/1357] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:08:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:08:19.192196 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c9fc3-b97c-4301-9fe4-1c5653b7c296 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:19.192875 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c9fc3-b97c-4301-9fe4-1c5653b7c296 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:19.194029 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c9fc3-b97c-4301-9fe4-1c5653b7c296 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:19.196226 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 690/1358] 10.222.0.41 () {54 vars in 961 bytes} [Fri Jun 3 16:08:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:08:19.203560 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2362d45-04f6-460d-9e2f-ca1efbeebdbe None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:19.203867 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2362d45-04f6-460d-9e2f-ca1efbeebdbe None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:19.204145 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2362d45-04f6-460d-9e2f-ca1efbeebdbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:19.296868 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:08:19.307868 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-d2362d45-04f6-460d-9e2f-ca1efbeebdbe None None] MFA Rules not processed for user `2fd08ea39ee94b92ac6763608e69f158`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:08:19.309044 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d2362d45-04f6-460d-9e2f-ca1efbeebdbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:08:19.403444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 669/1359] 10.222.0.41 () {58 vars in 1058 bytes} [Fri Jun 3 16:08:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:08:19.409621 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c9fc3-b97c-4301-9fe4-1c5653b7c296 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:08:19.425330 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c9fc3-b97c-4301-9fe4-1c5653b7c296 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:08:19.431822 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd04660-3e46-4091-9c94-0332a2d4f013 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:08:19.432476 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd04660-3e46-4091-9c94-0332a2d4f013 service nova] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:19.432761 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd04660-3e46-4091-9c94-0332a2d4f013 service nova] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:19.433253 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd04660-3e46-4091-9c94-0332a2d4f013 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:19.444611 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd04660-3e46-4091-9c94-0332a2d4f013 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:08:19.446665 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd04660-3e46-4091-9c94-0332a2d4f013 service nova] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:08:19.458216 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 691/1360] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:08:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:08:21.935360 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67a823-421f-414f-88c1-27b0a33479c4 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:21.935881 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67a823-421f-414f-88c1-27b0a33479c4 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:21.935975 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67a823-421f-414f-88c1-27b0a33479c4 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:21.937439 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 670/1361] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:08:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:08:21.942428 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa26f2c-78fe-4059-8424-345ef525e5bc None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:21.942863 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa26f2c-78fe-4059-8424-345ef525e5bc None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:21.942990 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa26f2c-78fe-4059-8424-345ef525e5bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:21.962273 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-eaa26f2c-78fe-4059-8424-345ef525e5bc None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:08:21.963570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-eaa26f2c-78fe-4059-8424-345ef525e5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:08:21.978375 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 692/1362] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:08:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:08:22.015456 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67a823-421f-414f-88c1-27b0a33479c4 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:08:22.034533 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67a823-421f-414f-88c1-27b0a33479c4 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:08:22.040840 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863c58e-c37d-446d-b49f-ecda0a49402a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:08:22.041478 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863c58e-c37d-446d-b49f-ecda0a49402a service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:22.041771 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863c58e-c37d-446d-b49f-ecda0a49402a service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:22.042041 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863c58e-c37d-446d-b49f-ecda0a49402a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:22.051271 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e863c58e-c37d-446d-b49f-ecda0a49402a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:08:22.053410 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e863c58e-c37d-446d-b49f-ecda0a49402a service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:08:22.063343 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 671/1363] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:08:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:08:24.666594 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1e5af-753f-45c3-aff1-3cfb1ba053ba None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:24.667084 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1e5af-753f-45c3-aff1-3cfb1ba053ba None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:24.667084 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1e5af-753f-45c3-aff1-3cfb1ba053ba None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:24.668483 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 693/1364] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:08:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:08:24.673625 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de249d53-0af4-4d12-aa0f-d8a5a9fbe9c8 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:24.674026 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de249d53-0af4-4d12-aa0f-d8a5a9fbe9c8 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:24.674185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de249d53-0af4-4d12-aa0f-d8a5a9fbe9c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:24.696643 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-de249d53-0af4-4d12-aa0f-d8a5a9fbe9c8 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:08:24.698750 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-de249d53-0af4-4d12-aa0f-d8a5a9fbe9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:08:24.708234 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 672/1365] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:08:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:08:24.753660 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de48ca-918e-4a79-8e10-eae1d1b11173 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:24.753980 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de48ca-918e-4a79-8e10-eae1d1b11173 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:24.754261 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de48ca-918e-4a79-8e10-eae1d1b11173 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:24.755829 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 694/1366] 10.222.0.41 () {54 vars in 961 bytes} [Fri Jun 3 16:08:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:08:24.761072 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d589b59-69f3-4fb7-bf3c-05759d1ca940 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:24.761366 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d589b59-69f3-4fb7-bf3c-05759d1ca940 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:24.761647 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d589b59-69f3-4fb7-bf3c-05759d1ca940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:24.782305 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-9d589b59-69f3-4fb7-bf3c-05759d1ca940 None None] MFA Rules not processed for user `2fd08ea39ee94b92ac6763608e69f158`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:08:24.783539 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9d589b59-69f3-4fb7-bf3c-05759d1ca940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:08:24.793626 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 673/1367] 10.222.0.41 () {58 vars in 1058 bytes} [Fri Jun 3 16:08:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:08:24.801713 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de48ca-918e-4a79-8e10-eae1d1b11173 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:08:24.816237 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de48ca-918e-4a79-8e10-eae1d1b11173 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:08:24.821604 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90727014-ec2c-4e2c-81a5-b39b7a032e96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:08:24.822239 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90727014-ec2c-4e2c-81a5-b39b7a032e96 service nova] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:24.822524 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90727014-ec2c-4e2c-81a5-b39b7a032e96 service nova] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:24.822798 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90727014-ec2c-4e2c-81a5-b39b7a032e96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:24.830787 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90727014-ec2c-4e2c-81a5-b39b7a032e96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:08:24.832393 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90727014-ec2c-4e2c-81a5-b39b7a032e96 service nova] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:08:24.842472 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 695/1368] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:08:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:08:27.191628 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da8fef-0e5a-4b4d-8f8f-b476023ce71c None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:27.192186 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da8fef-0e5a-4b4d-8f8f-b476023ce71c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:27.192186 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da8fef-0e5a-4b4d-8f8f-b476023ce71c None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:27.193590 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 674/1369] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:08:27.199350 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47401a0-d104-4d09-a408-9055705f1eb6 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:27.200177 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47401a0-d104-4d09-a408-9055705f1eb6 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:27.200533 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47401a0-d104-4d09-a408-9055705f1eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:27.218444 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-b47401a0-d104-4d09-a408-9055705f1eb6 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:08:27.219777 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-b47401a0-d104-4d09-a408-9055705f1eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:08:27.229986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 696/1370] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:08:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:08:27.278271 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da8fef-0e5a-4b4d-8f8f-b476023ce71c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:08:27.294014 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da8fef-0e5a-4b4d-8f8f-b476023ce71c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:08:27.299671 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbbe4f3-7e5f-4579-8f61-d0fc783803e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:08:27.300895 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbe4f3-7e5f-4579-8f61-d0fc783803e9 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:27.301258 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbe4f3-7e5f-4579-8f61-d0fc783803e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:27.301612 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbe4f3-7e5f-4579-8f61-d0fc783803e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:27.312113 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbbe4f3-7e5f-4579-8f61-d0fc783803e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:08:27.313746 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbbe4f3-7e5f-4579-8f61-d0fc783803e9 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:08:27.324970 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 675/1371] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:08:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:08:27.340261 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8220b16-196e-4f8d-9636-94842f52c6d4 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:27.340645 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8220b16-196e-4f8d-9636-94842f52c6d4 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:27.340878 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8220b16-196e-4f8d-9636-94842f52c6d4 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:27.342849 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 697/1372] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:08:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:08:27.348300 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47081ff-31e7-436b-8f77-f78cfa9b8f67 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:27.348610 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47081ff-31e7-436b-8f77-f78cfa9b8f67 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:27.349037 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47081ff-31e7-436b-8f77-f78cfa9b8f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:27.368342 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c47081ff-31e7-436b-8f77-f78cfa9b8f67 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:08:27.369532 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c47081ff-31e7-436b-8f77-f78cfa9b8f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:08:27.377846 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 676/1373] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:08:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:08:27.687802 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1458a-24d6-4ce5-8604-8720c627ce90 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:27.688005 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1458a-24d6-4ce5-8604-8720c627ce90 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:27.688191 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1458a-24d6-4ce5-8604-8720c627ce90 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:27.689715 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 698/1374] 10.222.0.41 () {54 vars in 963 bytes} [Fri Jun 3 16:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:08:27.695242 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf8cdab-54d0-44b4-ac17-8632ae6a8513 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:27.695651 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf8cdab-54d0-44b4-ac17-8632ae6a8513 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:27.696000 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf8cdab-54d0-44b4-ac17-8632ae6a8513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:27.749574 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-1bf8cdab-54d0-44b4-ac17-8632ae6a8513 None None] MFA Rules not processed for user `62a3bbb9720049b29eb37b0dd88bbad7`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:08:27.750893 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1bf8cdab-54d0-44b4-ac17-8632ae6a8513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:08:27.817698 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 677/1375] 10.222.0.41 () {58 vars in 1060 bytes} [Fri Jun 3 16:08:27 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:08:27.825933 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce1458a-24d6-4ce5-8604-8720c627ce90 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:08:27.836899 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce1458a-24d6-4ce5-8604-8720c627ce90 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:08:27.840455 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe7045b-79cf-49c4-bf84-02ce59ca6a81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:08:27.841093 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7045b-79cf-49c4-bf84-02ce59ca6a81 service cinder] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:27.841375 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7045b-79cf-49c4-bf84-02ce59ca6a81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:27.841647 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7045b-79cf-49c4-bf84-02ce59ca6a81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:27.851317 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe7045b-79cf-49c4-bf84-02ce59ca6a81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:08:27.852729 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe7045b-79cf-49c4-bf84-02ce59ca6a81 service cinder] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:08:27.862767 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 699/1376] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:08:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:08:29.623974 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94a519-402d-4514-aff2-57d1b75aaa6f None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:29.624512 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94a519-402d-4514-aff2-57d1b75aaa6f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:29.624629 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94a519-402d-4514-aff2-57d1b75aaa6f None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:29.625934 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 678/1377] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:08:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:08:29.631957 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3fecf6-6232-4266-b7f0-591ce0a6548b None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:29.632596 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3fecf6-6232-4266-b7f0-591ce0a6548b None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:29.632596 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3fecf6-6232-4266-b7f0-591ce0a6548b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:29.650928 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-8d3fecf6-6232-4266-b7f0-591ce0a6548b None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:08:29.652213 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-8d3fecf6-6232-4266-b7f0-591ce0a6548b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:08:29.660398 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 700/1378] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:08:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:08:29.706427 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc94a519-402d-4514-aff2-57d1b75aaa6f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:08:29.722430 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc94a519-402d-4514-aff2-57d1b75aaa6f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:08:29.728545 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e01ea74-a422-4446-851f-720673a5f0a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:08:29.729207 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01ea74-a422-4446-851f-720673a5f0a6 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:29.729495 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01ea74-a422-4446-851f-720673a5f0a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:29.729776 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01ea74-a422-4446-851f-720673a5f0a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:29.739028 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e01ea74-a422-4446-851f-720673a5f0a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:08:29.740684 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e01ea74-a422-4446-851f-720673a5f0a6 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:08:29.745207 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:08:29.752926 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 679/1379] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:08:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:08:32.044533 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82d354-a4e6-4a15-aacc-80981c7ff50a None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:32.045441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82d354-a4e6-4a15-aacc-80981c7ff50a None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:32.045876 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82d354-a4e6-4a15-aacc-80981c7ff50a None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:32.048875 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 701/1380] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:08:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:08:32.052696 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5afbc-28e4-4609-af1f-1d0514ceaae2 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:32.053111 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5afbc-28e4-4609-af1f-1d0514ceaae2 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:32.053268 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5afbc-28e4-4609-af1f-1d0514ceaae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:32.071998 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-13b5afbc-28e4-4609-af1f-1d0514ceaae2 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:08:32.073410 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-13b5afbc-28e4-4609-af1f-1d0514ceaae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:08:32.082364 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 680/1381] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:08:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:08:32.127032 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82d354-a4e6-4a15-aacc-80981c7ff50a None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:08:32.143573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82d354-a4e6-4a15-aacc-80981c7ff50a None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:08:32.150019 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9191daf3-2fbf-4061-9e42-2237b6917267 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:08:32.150762 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191daf3-2fbf-4061-9e42-2237b6917267 service nova] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:32.151115 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191daf3-2fbf-4061-9e42-2237b6917267 service nova] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:32.151549 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191daf3-2fbf-4061-9e42-2237b6917267 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:32.160601 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9191daf3-2fbf-4061-9e42-2237b6917267 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:08:32.162259 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9191daf3-2fbf-4061-9e42-2237b6917267 service nova] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:08:32.172460 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 702/1382] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:08:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:08:34.468128 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91da87e-05e4-4680-bf2a-2f4b0e4a10a1 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:34.468692 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91da87e-05e4-4680-bf2a-2f4b0e4a10a1 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:34.468692 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91da87e-05e4-4680-bf2a-2f4b0e4a10a1 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:34.470088 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 681/1383] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:08:34.476079 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b74550-ef15-4455-aa17-b635ca3f22a7 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:34.476534 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b74550-ef15-4455-aa17-b635ca3f22a7 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:34.476644 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b74550-ef15-4455-aa17-b635ca3f22a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:34.494179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-69b74550-ef15-4455-aa17-b635ca3f22a7 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:08:34.495380 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-69b74550-ef15-4455-aa17-b635ca3f22a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:08:34.503328 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 703/1384] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:08:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:08:34.547251 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91da87e-05e4-4680-bf2a-2f4b0e4a10a1 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:08:34.569046 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91da87e-05e4-4680-bf2a-2f4b0e4a10a1 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:08:34.574543 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d168ff8e-11b3-4dea-b7dc-295789371f3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:08:34.575206 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168ff8e-11b3-4dea-b7dc-295789371f3c service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:34.575521 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168ff8e-11b3-4dea-b7dc-295789371f3c service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:34.575796 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168ff8e-11b3-4dea-b7dc-295789371f3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:34.585093 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d168ff8e-11b3-4dea-b7dc-295789371f3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:08:34.586680 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d168ff8e-11b3-4dea-b7dc-295789371f3c service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:08:34.596491 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 682/1385] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:08:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:08:37.065293 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb68735-3af1-4cd3-8a90-a929b81c7197 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:37.065821 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb68735-3af1-4cd3-8a90-a929b81c7197 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:37.065821 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb68735-3af1-4cd3-8a90-a929b81c7197 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:37.067262 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 704/1386] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:08:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:08:37.072778 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c75ce4-1b65-48a6-8379-6babbc241583 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:37.073314 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c75ce4-1b65-48a6-8379-6babbc241583 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:37.073503 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c75ce4-1b65-48a6-8379-6babbc241583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:37.092195 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c4c75ce4-1b65-48a6-8379-6babbc241583 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:08:37.093435 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c4c75ce4-1b65-48a6-8379-6babbc241583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:08:37.102289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 683/1387] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:08:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:08:37.147830 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb68735-3af1-4cd3-8a90-a929b81c7197 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:08:37.161900 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb68735-3af1-4cd3-8a90-a929b81c7197 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:08:37.167806 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5594ce-65bc-4bb6-9221-47a74ec6ee55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:08:37.168446 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5594ce-65bc-4bb6-9221-47a74ec6ee55 service nova] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:37.168729 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5594ce-65bc-4bb6-9221-47a74ec6ee55 service nova] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:37.169008 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5594ce-65bc-4bb6-9221-47a74ec6ee55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:37.177089 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5594ce-65bc-4bb6-9221-47a74ec6ee55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:08:37.179226 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5594ce-65bc-4bb6-9221-47a74ec6ee55 service nova] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:08:37.188912 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 705/1388] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:08:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:08:39.443764 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70feea-9da2-43eb-9967-b9c226dd0405 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:39.444626 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70feea-9da2-43eb-9967-b9c226dd0405 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:39.444924 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70feea-9da2-43eb-9967-b9c226dd0405 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:39.446325 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 684/1389] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:08:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:08:39.453389 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957135ba-275c-4e17-b594-f325addaf395 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:39.453816 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957135ba-275c-4e17-b594-f325addaf395 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:39.454131 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957135ba-275c-4e17-b594-f325addaf395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:39.477629 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-957135ba-275c-4e17-b594-f325addaf395 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:08:39.479183 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-957135ba-275c-4e17-b594-f325addaf395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:08:39.489406 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 706/1390] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:08:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:08:39.545666 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70feea-9da2-43eb-9967-b9c226dd0405 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:08:39.559584 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70feea-9da2-43eb-9967-b9c226dd0405 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:08:39.564991 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ddedc-7af2-4817-aa98-8a57be288754 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:08:39.565621 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ddedc-7af2-4817-aa98-8a57be288754 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:39.565901 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ddedc-7af2-4817-aa98-8a57be288754 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:39.566181 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ddedc-7af2-4817-aa98-8a57be288754 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:39.574641 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245ddedc-7af2-4817-aa98-8a57be288754 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:08:39.576351 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245ddedc-7af2-4817-aa98-8a57be288754 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:08:39.586879 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 685/1391] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:08:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:08:41.986949 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5266b4-f0d4-4d77-b9e7-9ce8f43d5914 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:41.987577 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5266b4-f0d4-4d77-b9e7-9ce8f43d5914 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:41.987577 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5266b4-f0d4-4d77-b9e7-9ce8f43d5914 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:41.988956 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 707/1392] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:08:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:08:41.994854 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684f1ee-504e-4425-84a5-66a8bb8a467a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:41.995320 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684f1ee-504e-4425-84a5-66a8bb8a467a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:41.995438 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684f1ee-504e-4425-84a5-66a8bb8a467a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:42.014321 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-e684f1ee-504e-4425-84a5-66a8bb8a467a None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:08:42.015529 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e684f1ee-504e-4425-84a5-66a8bb8a467a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:08:42.024713 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 686/1393] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:08:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:08:42.069271 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5266b4-f0d4-4d77-b9e7-9ce8f43d5914 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:08:42.083712 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5266b4-f0d4-4d77-b9e7-9ce8f43d5914 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:08:42.089250 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7bb9a3-2800-4aaa-a740-d4ed97c0cf9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:08:42.089920 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bb9a3-2800-4aaa-a740-d4ed97c0cf9a service nova] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:42.090230 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bb9a3-2800-4aaa-a740-d4ed97c0cf9a service nova] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:42.090491 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bb9a3-2800-4aaa-a740-d4ed97c0cf9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:42.098847 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7bb9a3-2800-4aaa-a740-d4ed97c0cf9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:08:42.100964 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7bb9a3-2800-4aaa-a740-d4ed97c0cf9a service nova] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:08:42.110315 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 708/1394] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:08:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:08:43.770852 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966b842-11aa-4a2f-88a1-d20b1b5ea616 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:43.771445 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966b842-11aa-4a2f-88a1-d20b1b5ea616 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:43.771562 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966b842-11aa-4a2f-88a1-d20b1b5ea616 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:43.773095 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 687/1395] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:08:43.778413 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d54253-48a4-4e90-9ddc-fca5ae5316cd None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:43.778836 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d54253-48a4-4e90-9ddc-fca5ae5316cd None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:43.778985 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d54253-48a4-4e90-9ddc-fca5ae5316cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:43.785109 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=52 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:08:43.798045 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-70d54253-48a4-4e90-9ddc-fca5ae5316cd None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:08:43.799235 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-70d54253-48a4-4e90-9ddc-fca5ae5316cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:08:43.807656 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 709/1396] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:08:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:08:43.863535 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27166622-2e60-4c29-bf59-40cc8938629c None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:43.863933 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27166622-2e60-4c29-bf59-40cc8938629c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:43.864231 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27166622-2e60-4c29-bf59-40cc8938629c None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:43.866988 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 688/1397] 10.222.0.41 () {54 vars in 962 bytes} [Fri Jun 3 16:08:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:08:43.872391 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94531e-0e8c-4eeb-9609-ae0110ba43a1 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:43.872821 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94531e-0e8c-4eeb-9609-ae0110ba43a1 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:43.873189 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94531e-0e8c-4eeb-9609-ae0110ba43a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:43.920768 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-7f94531e-0e8c-4eeb-9609-ae0110ba43a1 None None] MFA Rules not processed for user `bbaf66d6e1f54047917cc615546bbd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:08:43.921964 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-7f94531e-0e8c-4eeb-9609-ae0110ba43a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:08:43.983886 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 710/1398] 10.222.0.41 () {58 vars in 1059 bytes} [Fri Jun 3 16:08:43 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:08:43.991432 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27166622-2e60-4c29-bf59-40cc8938629c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:08:44.003038 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27166622-2e60-4c29-bf59-40cc8938629c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:08:44.006563 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86779c-029f-40b3-89c5-fd8d302ecb95 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbaf66d6e1f54047917cc615546bbd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:08:44.007200 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86779c-029f-40b3-89c5-fd8d302ecb95 service glance] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:44.007525 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86779c-029f-40b3-89c5-fd8d302ecb95 service glance] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:44.007827 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86779c-029f-40b3-89c5-fd8d302ecb95 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:44.017383 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a86779c-029f-40b3-89c5-fd8d302ecb95 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:08:44.018858 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a86779c-029f-40b3-89c5-fd8d302ecb95 service glance] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:08:44.030919 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 689/1399] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:08:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 16:08:44.363958 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab1558-37f9-48f2-9384-b069c072c777 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:44.364253 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab1558-37f9-48f2-9384-b069c072c777 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:44.364536 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab1558-37f9-48f2-9384-b069c072c777 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:44.365961 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 711/1400] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:08:44.371280 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a11535-2516-4640-977e-480d6c2b7c73 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:44.371608 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a11535-2516-4640-977e-480d6c2b7c73 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:44.371883 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a11535-2516-4640-977e-480d6c2b7c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:44.389602 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-45a11535-2516-4640-977e-480d6c2b7c73 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:08:44.390763 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-45a11535-2516-4640-977e-480d6c2b7c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:08:44.399162 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 690/1401] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:08:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:08:44.442096 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab1558-37f9-48f2-9384-b069c072c777 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:08:44.456491 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab1558-37f9-48f2-9384-b069c072c777 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:08:44.462391 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff319f3-3eba-4a8c-a605-9f6d73f2f433 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:08:44.463014 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff319f3-3eba-4a8c-a605-9f6d73f2f433 service nova] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:44.463379 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff319f3-3eba-4a8c-a605-9f6d73f2f433 service nova] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:44.463594 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff319f3-3eba-4a8c-a605-9f6d73f2f433 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:44.471671 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff319f3-3eba-4a8c-a605-9f6d73f2f433 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:08:44.473227 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff319f3-3eba-4a8c-a605-9f6d73f2f433 service nova] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:08:44.482508 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 712/1402] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:08:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:08:44.565456 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60903a11-e29a-4abc-bb23-2b5cab5e1d88 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:44.565745 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60903a11-e29a-4abc-bb23-2b5cab5e1d88 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:44.566024 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60903a11-e29a-4abc-bb23-2b5cab5e1d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:44.620024 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-60903a11-e29a-4abc-bb23-2b5cab5e1d88 None None] MFA Rules not processed for user `875140cf51d9410da45f8c3805b26204`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:08:44.621247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-60903a11-e29a-4abc-bb23-2b5cab5e1d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:08:44.690083 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 691/1403] 10.222.0.41 () {58 vars in 1009 bytes} [Fri Jun 3 16:08:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:08:44.736796 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710fb88-f1e8-4f32-a275-2504dc58ee09 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:44.737087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710fb88-f1e8-4f32-a275-2504dc58ee09 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:44.737364 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710fb88-f1e8-4f32-a275-2504dc58ee09 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:44.738756 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 713/1404] 10.222.0.41 () {54 vars in 942 bytes} [Fri Jun 3 16:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:08:44.746516 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbfd81b-de26-4e65-b02b-70dcc5cc441f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:44.746804 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbfd81b-de26-4e65-b02b-70dcc5cc441f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:44.747079 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbfd81b-de26-4e65-b02b-70dcc5cc441f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:44.800629 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-fdbfd81b-de26-4e65-b02b-70dcc5cc441f None None] MFA Rules not processed for user `6c5f513b781a4a73b443cded13359eda`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:08:44.802069 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-fdbfd81b-de26-4e65-b02b-70dcc5cc441f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:08:44.835206 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:08:44.873908 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 692/1405] 10.222.0.41 () {58 vars in 1039 bytes} [Fri Jun 3 16:08:44 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:08:44.884167 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710fb88-f1e8-4f32-a275-2504dc58ee09 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:08:44.897161 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710fb88-f1e8-4f32-a275-2504dc58ee09 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:08:44.902325 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de1539-d185-4177-be66-4126ca1326e9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5f513b781a4a73b443cded13359eda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:08:44.902941 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de1539-d185-4177-be66-4126ca1326e9 service swift] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:44.903227 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de1539-d185-4177-be66-4126ca1326e9 service swift] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:44.903560 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de1539-d185-4177-be66-4126ca1326e9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:44.912124 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de1539-d185-4177-be66-4126ca1326e9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:08:44.913676 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de1539-d185-4177-be66-4126ca1326e9 service swift] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:08:44.923709 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 714/1406] 10.222.0.41 () {60 vars in 1384 bytes} [Fri Jun 3 16:08:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 16:08:46.789546 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649345d3-4b78-4f84-a865-4de26989d473 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:46.789932 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649345d3-4b78-4f84-a865-4de26989d473 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:46.790014 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649345d3-4b78-4f84-a865-4de26989d473 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:46.795431 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 693/1407] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:08:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:08:46.796776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832c9d6-fe4f-4290-915a-e2aa7874f076 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:46.797186 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832c9d6-fe4f-4290-915a-e2aa7874f076 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:46.797360 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832c9d6-fe4f-4290-915a-e2aa7874f076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:46.815737 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-8832c9d6-fe4f-4290-915a-e2aa7874f076 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:08:46.816929 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-8832c9d6-fe4f-4290-915a-e2aa7874f076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:08:46.826009 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 715/1408] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:08:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:08:46.872293 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649345d3-4b78-4f84-a865-4de26989d473 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:08:46.886757 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649345d3-4b78-4f84-a865-4de26989d473 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:08:46.892138 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf80d3-6dd1-46ec-a0dc-02e81e4bf452 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:08:46.892771 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf80d3-6dd1-46ec-a0dc-02e81e4bf452 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:08:46.893197 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf80d3-6dd1-46ec-a0dc-02e81e4bf452 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:08:46.895284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf80d3-6dd1-46ec-a0dc-02e81e4bf452 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:08:46.903907 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf80d3-6dd1-46ec-a0dc-02e81e4bf452 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:08:46.905522 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf80d3-6dd1-46ec-a0dc-02e81e4bf452 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:08:46.915962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 694/1409] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:08:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:08:56.810648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=55 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:08:56.910856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=21 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:09:08.386269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ee7a9-788d-479f-a9c3-7177a6d41363 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:08.386894 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ee7a9-788d-479f-a9c3-7177a6d41363 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:08.386894 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ee7a9-788d-479f-a9c3-7177a6d41363 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:08.388242 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 716/1410] 10.222.0.41 () {54 vars in 918 bytes} [Fri Jun 3 16:09:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:09:08.393344 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f995a71-ad7a-4016-885e-8865c5a72129 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:08.393805 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f995a71-ad7a-4016-885e-8865c5a72129 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:08.393910 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f995a71-ad7a-4016-885e-8865c5a72129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:08.412391 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-1f995a71-ad7a-4016-885e-8865c5a72129 None None] MFA Rules not processed for user `be901786f8934d559e0bac5d622f97cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:09:08.413584 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1f995a71-ad7a-4016-885e-8865c5a72129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:09:08.421970 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 695/1411] 10.222.0.41 () {58 vars in 1015 bytes} [Fri Jun 3 16:09:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:09:08.435523 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8a49b-7333-400f-882c-e0c1b9dd9603 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:08.435818 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8a49b-7333-400f-882c-e0c1b9dd9603 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:08.436099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8a49b-7333-400f-882c-e0c1b9dd9603 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:08.437478 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 717/1412] 10.222.0.41 () {54 vars in 918 bytes} [Fri Jun 3 16:09:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:09:08.441943 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6435c72-a1ba-475e-9c08-010ac077cb79 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:08.442233 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6435c72-a1ba-475e-9c08-010ac077cb79 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:08.442513 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6435c72-a1ba-475e-9c08-010ac077cb79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:08.460367 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-e6435c72-a1ba-475e-9c08-010ac077cb79 None None] MFA Rules not processed for user `2fd08ea39ee94b92ac6763608e69f158`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:09:08.462149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e6435c72-a1ba-475e-9c08-010ac077cb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:09:08.471035 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 696/1413] 10.222.0.41 () {58 vars in 1015 bytes} [Fri Jun 3 16:09:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:09:11.163886 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b9ce4-0fc8-4bd1-b200-a92c63b4f364 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:11.164430 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b9ce4-0fc8-4bd1-b200-a92c63b4f364 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:11.164430 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b9ce4-0fc8-4bd1-b200-a92c63b4f364 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:11.165838 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 718/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 16:09:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:09:11.171130 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92d112-f13d-41a3-8b8e-e620467b8bd1 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:11.171617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92d112-f13d-41a3-8b8e-e620467b8bd1 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:11.171716 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92d112-f13d-41a3-8b8e-e620467b8bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:11.190299 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-5e92d112-f13d-41a3-8b8e-e620467b8bd1 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:09:11.191538 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5e92d112-f13d-41a3-8b8e-e620467b8bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:09:11.199843 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 697/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 16:09:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:09:11.243953 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b9ce4-0fc8-4bd1-b200-a92c63b4f364 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:09:11.259908 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b9ce4-0fc8-4bd1-b200-a92c63b4f364 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:09:11.265287 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6456b-b406-4fc2-98b3-031bdace38f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:09:11.265930 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6456b-b406-4fc2-98b3-031bdace38f5 service nova] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:11.266217 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6456b-b406-4fc2-98b3-031bdace38f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:11.266499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6456b-b406-4fc2-98b3-031bdace38f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:11.274772 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6456b-b406-4fc2-98b3-031bdace38f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:09:11.276382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6456b-b406-4fc2-98b3-031bdace38f5 service nova] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:09:11.285580 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 719/1416] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:09:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:09:12.245294 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc48d40-bbf9-4c69-b80d-2db1f2783492 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:12.245897 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc48d40-bbf9-4c69-b80d-2db1f2783492 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:12.245897 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc48d40-bbf9-4c69-b80d-2db1f2783492 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:12.247192 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 698/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 3 16:09:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:09:12.252385 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e2185-04f9-4300-96c7-b9cddb5d92c7 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:12.252768 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e2185-04f9-4300-96c7-b9cddb5d92c7 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:12.252941 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e2185-04f9-4300-96c7-b9cddb5d92c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:12.270461 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-d03e2185-04f9-4300-96c7-b9cddb5d92c7 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:09:12.271673 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-d03e2185-04f9-4300-96c7-b9cddb5d92c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:09:12.279490 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 720/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 3 16:09:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:09:12.441181 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc48d40-bbf9-4c69-b80d-2db1f2783492 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:09:12.455227 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc48d40-bbf9-4c69-b80d-2db1f2783492 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:09:12.460496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a01d17-fcff-4bdc-85fb-eed1a57e890e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:09:12.461124 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a01d17-fcff-4bdc-85fb-eed1a57e890e service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:12.461408 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a01d17-fcff-4bdc-85fb-eed1a57e890e service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:12.462179 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a01d17-fcff-4bdc-85fb-eed1a57e890e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:12.470147 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a01d17-fcff-4bdc-85fb-eed1a57e890e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:09:12.471769 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a01d17-fcff-4bdc-85fb-eed1a57e890e service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:09:12.481533 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 699/1419] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:09:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:09:13.400347 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756745e6-d667-4974-b9f4-6b9d97d87896 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:13.400859 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756745e6-d667-4974-b9f4-6b9d97d87896 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:13.400975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756745e6-d667-4974-b9f4-6b9d97d87896 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:13.402233 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 721/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 3 16:09:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:09:13.407086 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80dfa6-272e-46f3-8d47-7033811b44c4 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:13.407512 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80dfa6-272e-46f3-8d47-7033811b44c4 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:13.407709 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80dfa6-272e-46f3-8d47-7033811b44c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:13.425721 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-5b80dfa6-272e-46f3-8d47-7033811b44c4 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:09:13.426912 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5b80dfa6-272e-46f3-8d47-7033811b44c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:09:13.434622 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 700/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 3 16:09:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:09:13.583254 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756745e6-d667-4974-b9f4-6b9d97d87896 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:09:13.597067 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756745e6-d667-4974-b9f4-6b9d97d87896 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:09:13.602829 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6ff06-202a-4784-8abd-40ed087df866 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:09:13.603485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6ff06-202a-4784-8abd-40ed087df866 service nova] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:13.603769 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6ff06-202a-4784-8abd-40ed087df866 service nova] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:13.604043 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6ff06-202a-4784-8abd-40ed087df866 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:13.611869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f6ff06-202a-4784-8abd-40ed087df866 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:09:13.613464 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f6ff06-202a-4784-8abd-40ed087df866 service nova] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:09:13.623080 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 722/1422] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:09:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:09:14.517593 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49992b-3808-43ca-a3d1-45c41d28517a None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:14.518124 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49992b-3808-43ca-a3d1-45c41d28517a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:14.518124 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49992b-3808-43ca-a3d1-45c41d28517a None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:14.519456 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 701/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 3 16:09:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:09:14.524958 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f83e3fd-2190-4a75-8053-66394475a307 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:14.525355 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f83e3fd-2190-4a75-8053-66394475a307 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:14.525511 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f83e3fd-2190-4a75-8053-66394475a307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:14.544036 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-4f83e3fd-2190-4a75-8053-66394475a307 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:09:14.545236 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-4f83e3fd-2190-4a75-8053-66394475a307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:09:14.552931 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 723/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 3 16:09:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:09:14.700280 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce49992b-3808-43ca-a3d1-45c41d28517a None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:09:14.714707 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce49992b-3808-43ca-a3d1-45c41d28517a None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:09:14.719992 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d26bf0-1909-4be5-bb78-33201a486b25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:09:14.720614 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d26bf0-1909-4be5-bb78-33201a486b25 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:14.720894 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d26bf0-1909-4be5-bb78-33201a486b25 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:14.721166 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d26bf0-1909-4be5-bb78-33201a486b25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:14.729532 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d26bf0-1909-4be5-bb78-33201a486b25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:09:14.731089 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d26bf0-1909-4be5-bb78-33201a486b25 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:09:14.740208 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 702/1425] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:09:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:09:15.644075 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606ed8e-d58c-41f1-8558-23b8e1ce13fe None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:15.644610 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606ed8e-d58c-41f1-8558-23b8e1ce13fe None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:15.644610 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606ed8e-d58c-41f1-8558-23b8e1ce13fe None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:15.645971 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 724/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 3 16:09:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:09:15.651216 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9113a1-7f7c-4c3c-b6d7-f6d9106dc40d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:15.651669 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9113a1-7f7c-4c3c-b6d7-f6d9106dc40d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:15.651810 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9113a1-7f7c-4c3c-b6d7-f6d9106dc40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:15.669240 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-3c9113a1-7f7c-4c3c-b6d7-f6d9106dc40d None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:09:15.670487 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3c9113a1-7f7c-4c3c-b6d7-f6d9106dc40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:09:15.678212 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 703/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 3 16:09:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:09:15.825563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606ed8e-d58c-41f1-8558-23b8e1ce13fe None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:09:15.839588 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606ed8e-d58c-41f1-8558-23b8e1ce13fe None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:09:15.845064 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8f98c-0005-4126-89c7-40b30d1143a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:09:15.845691 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8f98c-0005-4126-89c7-40b30d1143a7 service nova] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:15.845973 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8f98c-0005-4126-89c7-40b30d1143a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:15.846255 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8f98c-0005-4126-89c7-40b30d1143a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:15.854745 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e8f98c-0005-4126-89c7-40b30d1143a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:09:15.856376 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e8f98c-0005-4126-89c7-40b30d1143a7 service nova] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:09:15.865432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 725/1428] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:09:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:09:16.702768 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec1840-fd72-4cfd-bfb5-502523c97b46 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:16.703377 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec1840-fd72-4cfd-bfb5-502523c97b46 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:16.703377 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec1840-fd72-4cfd-bfb5-502523c97b46 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:16.704694 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 704/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 3 16:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:09:16.709617 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0ebd2-4b85-4334-9490-1bd9433e7787 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:16.709992 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0ebd2-4b85-4334-9490-1bd9433e7787 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:16.710181 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0ebd2-4b85-4334-9490-1bd9433e7787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:16.728282 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-58c0ebd2-4b85-4334-9490-1bd9433e7787 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:09:16.729516 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-58c0ebd2-4b85-4334-9490-1bd9433e7787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:09:16.737345 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 726/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 3 16:09:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:09:16.884507 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec1840-fd72-4cfd-bfb5-502523c97b46 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:09:16.898646 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec1840-fd72-4cfd-bfb5-502523c97b46 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:09:16.904847 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460ac0eb-cb52-4f5b-bb4d-0c01fa1fc967 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:09:16.905476 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ac0eb-cb52-4f5b-bb4d-0c01fa1fc967 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:16.905764 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ac0eb-cb52-4f5b-bb4d-0c01fa1fc967 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:16.906045 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ac0eb-cb52-4f5b-bb4d-0c01fa1fc967 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:16.914074 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460ac0eb-cb52-4f5b-bb4d-0c01fa1fc967 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:09:16.915685 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460ac0eb-cb52-4f5b-bb4d-0c01fa1fc967 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:09:16.924876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 705/1431] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:09:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:09:17.765739 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84811bba-bfe7-4892-b412-255394b9e56a None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:17.766274 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84811bba-bfe7-4892-b412-255394b9e56a None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:17.766371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84811bba-bfe7-4892-b412-255394b9e56a None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:17.767710 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 727/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 3 16:09:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:09:17.772588 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd2b41-ab9f-4bb7-ae3f-6dc46040c1ca None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:17.773001 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd2b41-ab9f-4bb7-ae3f-6dc46040c1ca None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:17.773177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd2b41-ab9f-4bb7-ae3f-6dc46040c1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:17.792074 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-9fcd2b41-ab9f-4bb7-ae3f-6dc46040c1ca None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:09:17.793252 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9fcd2b41-ab9f-4bb7-ae3f-6dc46040c1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:09:17.800848 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 706/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 3 16:09:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:09:17.947119 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84811bba-bfe7-4892-b412-255394b9e56a None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:09:17.961039 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84811bba-bfe7-4892-b412-255394b9e56a None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:09:17.966778 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad2d1b-169e-4975-a094-53055fe063bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:09:17.967430 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad2d1b-169e-4975-a094-53055fe063bc service nova] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:17.967716 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad2d1b-169e-4975-a094-53055fe063bc service nova] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:17.967989 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad2d1b-169e-4975-a094-53055fe063bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:17.975853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad2d1b-169e-4975-a094-53055fe063bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:09:17.978002 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad2d1b-169e-4975-a094-53055fe063bc service nova] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:09:17.987101 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 728/1434] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:09:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:09:18.827113 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cdb9a2-4d14-477e-bd89-115cedb980c1 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:18.827567 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cdb9a2-4d14-477e-bd89-115cedb980c1 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:18.827705 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cdb9a2-4d14-477e-bd89-115cedb980c1 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:18.829073 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 707/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 3 16:09:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:09:18.834050 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ccb961-3eb2-4f85-ac7e-07ba72e60467 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:18.834478 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ccb961-3eb2-4f85-ac7e-07ba72e60467 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:18.834636 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ccb961-3eb2-4f85-ac7e-07ba72e60467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:18.852639 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-08ccb961-3eb2-4f85-ac7e-07ba72e60467 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:09:18.853818 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-08ccb961-3eb2-4f85-ac7e-07ba72e60467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:09:18.861432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 729/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 3 16:09:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:09:19.010410 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cdb9a2-4d14-477e-bd89-115cedb980c1 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:09:19.025256 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cdb9a2-4d14-477e-bd89-115cedb980c1 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:09:19.031165 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b65efe-5c89-4055-ac77-28d4596e0f33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:09:19.031824 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b65efe-5c89-4055-ac77-28d4596e0f33 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:19.032109 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b65efe-5c89-4055-ac77-28d4596e0f33 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:19.032386 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b65efe-5c89-4055-ac77-28d4596e0f33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:19.040453 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b65efe-5c89-4055-ac77-28d4596e0f33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:09:19.042027 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b65efe-5c89-4055-ac77-28d4596e0f33 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:09:19.051317 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 708/1437] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:09:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:09:19.892204 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f380328-3383-458d-8811-39952be74d5c None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:19.892794 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f380328-3383-458d-8811-39952be74d5c None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:19.892794 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f380328-3383-458d-8811-39952be74d5c None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:19.894102 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 730/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 3 16:09:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:09:19.899131 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0300faf-7e80-47f9-b4f5-b50439c5891e None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:19.899837 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0300faf-7e80-47f9-b4f5-b50439c5891e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:19.900028 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0300faf-7e80-47f9-b4f5-b50439c5891e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:19.918134 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c0300faf-7e80-47f9-b4f5-b50439c5891e None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:09:19.919364 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c0300faf-7e80-47f9-b4f5-b50439c5891e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:09:19.927036 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 709/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 3 16:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:09:20.074358 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f380328-3383-458d-8811-39952be74d5c None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:09:20.088843 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f380328-3383-458d-8811-39952be74d5c None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:09:20.094044 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4000e-40ab-433a-99ab-acc76d5d4705 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:09:20.094672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4000e-40ab-433a-99ab-acc76d5d4705 service nova] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:20.094953 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4000e-40ab-433a-99ab-acc76d5d4705 service nova] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:20.095232 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4000e-40ab-433a-99ab-acc76d5d4705 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:20.103877 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4000e-40ab-433a-99ab-acc76d5d4705 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:09:20.105431 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4000e-40ab-433a-99ab-acc76d5d4705 service nova] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:09:20.114526 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 731/1440] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:09:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:09:22.219594 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e62334-bdfd-456b-80e3-786e5a95924c None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:22.220162 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e62334-bdfd-456b-80e3-786e5a95924c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:22.220162 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e62334-bdfd-456b-80e3-786e5a95924c None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:22.221504 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 710/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 16:09:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:09:22.226844 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb0918-230a-4632-981e-28d77eee2c0f None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:22.227329 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb0918-230a-4632-981e-28d77eee2c0f None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:22.227450 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb0918-230a-4632-981e-28d77eee2c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:22.245239 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-61bb0918-230a-4632-981e-28d77eee2c0f None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:09:22.246427 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-61bb0918-230a-4632-981e-28d77eee2c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:09:22.254176 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 732/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 16:09:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:09:22.293353 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e62334-bdfd-456b-80e3-786e5a95924c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:09:22.303841 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e62334-bdfd-456b-80e3-786e5a95924c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:09:22.307781 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc74c7-3f58-419c-b9f1-40dd3e3e3569 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:09:22.308405 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc74c7-3f58-419c-b9f1-40dd3e3e3569 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:22.308686 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc74c7-3f58-419c-b9f1-40dd3e3e3569 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:22.308954 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc74c7-3f58-419c-b9f1-40dd3e3e3569 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:22.317457 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc74c7-3f58-419c-b9f1-40dd3e3e3569 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:09:22.319422 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc74c7-3f58-419c-b9f1-40dd3e3e3569 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:09:22.329082 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 711/1443] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:09:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:09:24.672405 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e13b350-d2cf-4a37-a44d-232b12be1d1c None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:24.672960 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e13b350-d2cf-4a37-a44d-232b12be1d1c None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:24.672960 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e13b350-d2cf-4a37-a44d-232b12be1d1c None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:24.674303 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 733/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 16:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:09:24.679397 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85a1fa-fb5f-4ac1-be79-957d112fad49 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:24.679805 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85a1fa-fb5f-4ac1-be79-957d112fad49 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:24.679990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85a1fa-fb5f-4ac1-be79-957d112fad49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:24.698436 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-2a85a1fa-fb5f-4ac1-be79-957d112fad49 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:09:24.699659 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2a85a1fa-fb5f-4ac1-be79-957d112fad49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:09:24.707256 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 712/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 16:09:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:09:24.744480 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e13b350-d2cf-4a37-a44d-232b12be1d1c None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:09:24.754842 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e13b350-d2cf-4a37-a44d-232b12be1d1c None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:09:24.758274 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa53c67-0905-4308-a0f4-db7c21a64c55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:09:24.759554 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa53c67-0905-4308-a0f4-db7c21a64c55 service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:24.759834 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa53c67-0905-4308-a0f4-db7c21a64c55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:24.760107 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa53c67-0905-4308-a0f4-db7c21a64c55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:24.768774 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa53c67-0905-4308-a0f4-db7c21a64c55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:09:24.770176 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa53c67-0905-4308-a0f4-db7c21a64c55 service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:09:24.780335 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 734/1446] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:09:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:09:27.309313 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26433b94-43b9-41c5-a81a-40a331f23944 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:27.309783 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26433b94-43b9-41c5-a81a-40a331f23944 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:27.309899 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26433b94-43b9-41c5-a81a-40a331f23944 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:27.311255 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 713/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 16:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:09:27.316489 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22753803-a904-4a20-ac5b-12c007d1cb95 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:27.316936 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22753803-a904-4a20-ac5b-12c007d1cb95 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:27.317050 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22753803-a904-4a20-ac5b-12c007d1cb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:27.334442 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-22753803-a904-4a20-ac5b-12c007d1cb95 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:09:27.335669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-22753803-a904-4a20-ac5b-12c007d1cb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:09:27.343376 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 735/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 16:09:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:09:27.382009 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26433b94-43b9-41c5-a81a-40a331f23944 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:09:27.383702 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-26433b94-43b9-41c5-a81a-40a331f23944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:09:27.396216 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26433b94-43b9-41c5-a81a-40a331f23944 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:09:27.399676 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acbdde9-837d-4d42-aeae-aef75c0717b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:09:27.400553 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbdde9-837d-4d42-aeae-aef75c0717b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:27.400835 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbdde9-837d-4d42-aeae-aef75c0717b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:27.401106 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbdde9-837d-4d42-aeae-aef75c0717b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:27.409928 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acbdde9-837d-4d42-aeae-aef75c0717b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:09:27.411362 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acbdde9-837d-4d42-aeae-aef75c0717b3 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:09:27.421873 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 714/1449] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:09:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:09:29.795321 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1984993-8137-4afb-b737-bf38197ad4f5 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:29.795919 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1984993-8137-4afb-b737-bf38197ad4f5 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:29.795919 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1984993-8137-4afb-b737-bf38197ad4f5 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:29.797344 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 736/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 16:09:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:09:29.802744 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa5324-0d00-4227-844a-458d120a4b48 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:29.803181 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa5324-0d00-4227-844a-458d120a4b48 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:29.803284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa5324-0d00-4227-844a-458d120a4b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:29.822013 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-05fa5324-0d00-4227-844a-458d120a4b48 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:09:29.823205 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-05fa5324-0d00-4227-844a-458d120a4b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:09:29.830929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 715/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 16:09:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:09:29.869222 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1984993-8137-4afb-b737-bf38197ad4f5 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:09:29.879822 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1984993-8137-4afb-b737-bf38197ad4f5 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:09:29.883279 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b461e-9203-4056-8028-3446135ca1fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:09:29.883958 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b461e-9203-4056-8028-3446135ca1fc service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:29.884241 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b461e-9203-4056-8028-3446135ca1fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:29.884522 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b461e-9203-4056-8028-3446135ca1fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:29.893149 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810b461e-9203-4056-8028-3446135ca1fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:09:29.894557 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810b461e-9203-4056-8028-3446135ca1fc service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:09:29.904534 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 737/1452] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:09:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:09:32.195025 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f655e371-0984-4303-9699-c30fe9b196dc None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:32.195593 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f655e371-0984-4303-9699-c30fe9b196dc None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:32.195593 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f655e371-0984-4303-9699-c30fe9b196dc None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:32.197532 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 716/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 16:09:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:09:32.202875 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b8aacc-18aa-40c5-b929-588dd509c964 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:32.203251 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b8aacc-18aa-40c5-b929-588dd509c964 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:32.203502 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b8aacc-18aa-40c5-b929-588dd509c964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:32.220844 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-c8b8aacc-18aa-40c5-b929-588dd509c964 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:09:32.222037 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-c8b8aacc-18aa-40c5-b929-588dd509c964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:09:32.229814 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 738/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 16:09:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:09:32.268001 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f655e371-0984-4303-9699-c30fe9b196dc None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:09:32.278858 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f655e371-0984-4303-9699-c30fe9b196dc None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:09:32.282380 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51d0f0-fd88-4826-8e78-d88ea7097b7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:09:32.283013 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51d0f0-fd88-4826-8e78-d88ea7097b7c service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:32.283318 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51d0f0-fd88-4826-8e78-d88ea7097b7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:32.283607 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51d0f0-fd88-4826-8e78-d88ea7097b7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:32.292878 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c51d0f0-fd88-4826-8e78-d88ea7097b7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:09:32.294308 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c51d0f0-fd88-4826-8e78-d88ea7097b7c service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:09:32.303569 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 717/1455] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:09:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:09:34.624095 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc0768-a98b-431c-955f-ee08692d2412 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:34.624604 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc0768-a98b-431c-955f-ee08692d2412 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:34.624604 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc0768-a98b-431c-955f-ee08692d2412 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:34.625931 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 739/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 16:09:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:09:34.631247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76b2e9-5bcf-4317-a10b-793aa86f2ada None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:34.631710 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76b2e9-5bcf-4317-a10b-793aa86f2ada None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:34.631852 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76b2e9-5bcf-4317-a10b-793aa86f2ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:34.649735 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-fd76b2e9-5bcf-4317-a10b-793aa86f2ada None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:09:34.651465 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-fd76b2e9-5bcf-4317-a10b-793aa86f2ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:09:34.659055 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 718/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 16:09:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:09:34.696594 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc0768-a98b-431c-955f-ee08692d2412 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:09:34.707237 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc0768-a98b-431c-955f-ee08692d2412 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:09:34.710707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770c280-2702-4f16-9ea2-b42e9e9d4c93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:09:34.711362 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770c280-2702-4f16-9ea2-b42e9e9d4c93 service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:34.711650 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770c280-2702-4f16-9ea2-b42e9e9d4c93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:34.712475 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770c280-2702-4f16-9ea2-b42e9e9d4c93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:34.721236 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b770c280-2702-4f16-9ea2-b42e9e9d4c93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:09:34.722638 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b770c280-2702-4f16-9ea2-b42e9e9d4c93 service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:09:34.731822 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 740/1458] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:09:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:09:37.033514 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06f31e-bbea-43d5-a163-e3ae94841a56 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:37.034067 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06f31e-bbea-43d5-a163-e3ae94841a56 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:37.034177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06f31e-bbea-43d5-a163-e3ae94841a56 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:37.035534 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 719/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 16:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:09:37.041566 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454204f-a9b9-4919-b388-678ec4d6470c None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:37.041953 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454204f-a9b9-4919-b388-678ec4d6470c None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:37.042135 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454204f-a9b9-4919-b388-678ec4d6470c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:37.048625 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=70 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:09:37.061645 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-d454204f-a9b9-4919-b388-678ec4d6470c None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:09:37.062843 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-d454204f-a9b9-4919-b388-678ec4d6470c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:09:37.070549 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 741/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 16:09:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:09:37.107906 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06f31e-bbea-43d5-a163-e3ae94841a56 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:09:37.113357 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=73 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:09:37.120629 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06f31e-bbea-43d5-a163-e3ae94841a56 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:09:37.124196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c93ff-a17c-472d-bef9-e9075c8bbe4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:09:37.124826 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c93ff-a17c-472d-bef9-e9075c8bbe4d service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:37.125112 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c93ff-a17c-472d-bef9-e9075c8bbe4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:37.125387 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c93ff-a17c-472d-bef9-e9075c8bbe4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:37.134285 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c93ff-a17c-472d-bef9-e9075c8bbe4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:09:37.135787 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c93ff-a17c-472d-bef9-e9075c8bbe4d service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:09:37.145046 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 720/1461] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:09:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:09:39.402728 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802bcb78-4414-436b-9d36-d25199fdc00a None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:39.403251 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802bcb78-4414-436b-9d36-d25199fdc00a None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:39.403251 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802bcb78-4414-436b-9d36-d25199fdc00a None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:39.404793 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 742/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 16:09:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:09:39.410166 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51a08f-71f3-40c5-b59e-181f8bfd66ec None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:39.410589 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51a08f-71f3-40c5-b59e-181f8bfd66ec None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:39.410730 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51a08f-71f3-40c5-b59e-181f8bfd66ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:39.428484 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-1c51a08f-71f3-40c5-b59e-181f8bfd66ec None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:09:39.429676 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1c51a08f-71f3-40c5-b59e-181f8bfd66ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:09:39.437939 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 721/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 16:09:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:09:39.474595 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802bcb78-4414-436b-9d36-d25199fdc00a None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:09:39.485760 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802bcb78-4414-436b-9d36-d25199fdc00a None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:09:39.489230 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215bcfa-312a-446c-9fdc-49bdbac08b7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:09:39.489890 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215bcfa-312a-446c-9fdc-49bdbac08b7c service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:39.490174 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215bcfa-312a-446c-9fdc-49bdbac08b7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:39.490452 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215bcfa-312a-446c-9fdc-49bdbac08b7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:39.499067 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6215bcfa-312a-446c-9fdc-49bdbac08b7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:09:39.500515 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6215bcfa-312a-446c-9fdc-49bdbac08b7c service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:09:39.509733 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 743/1464] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:09:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:09:41.762775 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e876a95-73d3-4609-af1a-6c476ab16118 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:41.763368 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e876a95-73d3-4609-af1a-6c476ab16118 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:41.763368 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e876a95-73d3-4609-af1a-6c476ab16118 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:41.764723 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 722/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 16:09:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:09:41.770073 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df144584-d9e2-4c0a-881f-1b1670906f88 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:41.770557 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df144584-d9e2-4c0a-881f-1b1670906f88 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:41.770664 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df144584-d9e2-4c0a-881f-1b1670906f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:41.788442 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-df144584-d9e2-4c0a-881f-1b1670906f88 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:09:41.789651 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-df144584-d9e2-4c0a-881f-1b1670906f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:09:41.797370 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 744/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 16:09:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:09:41.833926 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e876a95-73d3-4609-af1a-6c476ab16118 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:09:41.844415 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e876a95-73d3-4609-af1a-6c476ab16118 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:09:41.847936 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e720e668-0d9b-4ced-af0a-43d67a7afb26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:09:41.848572 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e720e668-0d9b-4ced-af0a-43d67a7afb26 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:41.848861 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e720e668-0d9b-4ced-af0a-43d67a7afb26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:41.849138 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e720e668-0d9b-4ced-af0a-43d67a7afb26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:41.858194 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e720e668-0d9b-4ced-af0a-43d67a7afb26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:09:41.859658 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e720e668-0d9b-4ced-af0a-43d67a7afb26 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:09:41.869435 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 723/1467] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:09:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:09:44.124522 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808319d-817e-49d8-9f9c-afef3c7eef0f None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:44.125088 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808319d-817e-49d8-9f9c-afef3c7eef0f None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:44.125088 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808319d-817e-49d8-9f9c-afef3c7eef0f None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:44.126427 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 745/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 16:09:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:09:44.131703 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34864647-b066-4a4b-86c5-b8897bc69a64 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:44.132128 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34864647-b066-4a4b-86c5-b8897bc69a64 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:44.132240 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34864647-b066-4a4b-86c5-b8897bc69a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:44.149977 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-34864647-b066-4a4b-86c5-b8897bc69a64 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:09:44.151462 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-34864647-b066-4a4b-86c5-b8897bc69a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:09:44.159033 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 724/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 16:09:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:09:44.195626 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9808319d-817e-49d8-9f9c-afef3c7eef0f None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:09:44.206941 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9808319d-817e-49d8-9f9c-afef3c7eef0f None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:09:44.210394 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8841fdbb-0d09-417d-94fa-52c0ad4e3743 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:09:44.211016 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841fdbb-0d09-417d-94fa-52c0ad4e3743 service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:09:44.211317 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841fdbb-0d09-417d-94fa-52c0ad4e3743 service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:09:44.211607 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841fdbb-0d09-417d-94fa-52c0ad4e3743 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:09:44.220212 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8841fdbb-0d09-417d-94fa-52c0ad4e3743 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:09:44.221608 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8841fdbb-0d09-417d-94fa-52c0ad4e3743 service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:09:44.230979 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 746/1470] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:09:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:09:54.144940 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=17 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:09:54.227504 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=17 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:10:15.831844 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4641181-8871-4866-93a2-9d4b36abd23f None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:10:15.832465 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4641181-8871-4866-93a2-9d4b36abd23f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:10:15.832465 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4641181-8871-4866-93a2-9d4b36abd23f None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:10:15.833782 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 725/1471] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:10:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:10:15.838952 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6d91a-2270-4712-878b-31f69f882f8a None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:10:15.839443 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6d91a-2270-4712-878b-31f69f882f8a None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:10:15.839556 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6d91a-2270-4712-878b-31f69f882f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:10:15.857439 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-ffb6d91a-2270-4712-878b-31f69f882f8a None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:10:15.858638 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-ffb6d91a-2270-4712-878b-31f69f882f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:10:15.866646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 747/1472] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:10:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:10:15.917752 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9e0c3-49a8-4c32-bcde-7446e0f775ab None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:10:15.918047 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9e0c3-49a8-4c32-bcde-7446e0f775ab None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:10:15.918332 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9e0c3-49a8-4c32-bcde-7446e0f775ab None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:10:15.919738 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 726/1473] 10.222.0.41 () {54 vars in 962 bytes} [Fri Jun 3 16:10:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:10:15.924675 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea69be4-31cb-4f11-b1f5-c04aa5662b36 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:10:15.924968 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea69be4-31cb-4f11-b1f5-c04aa5662b36 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:10:15.925250 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea69be4-31cb-4f11-b1f5-c04aa5662b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:10:15.943420 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-cea69be4-31cb-4f11-b1f5-c04aa5662b36 None None] MFA Rules not processed for user `bbaf66d6e1f54047917cc615546bbd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:10:15.944601 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-cea69be4-31cb-4f11-b1f5-c04aa5662b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:10:15.950854 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 748/1474] 10.222.0.41 () {58 vars in 1059 bytes} [Fri Jun 3 16:10:15 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:10:15.958095 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb9e0c3-49a8-4c32-bcde-7446e0f775ab None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:10:15.968511 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb9e0c3-49a8-4c32-bcde-7446e0f775ab None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:10:15.972025 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96680751-82ac-457f-897f-1159617aa8c9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbaf66d6e1f54047917cc615546bbd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:10:15.972650 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96680751-82ac-457f-897f-1159617aa8c9 service glance] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:10:15.972944 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96680751-82ac-457f-897f-1159617aa8c9 service glance] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:10:15.973226 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96680751-82ac-457f-897f-1159617aa8c9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:10:15.982405 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96680751-82ac-457f-897f-1159617aa8c9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:10:15.983938 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96680751-82ac-457f-897f-1159617aa8c9 service glance] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:10:15.993151 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 727/1475] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:10:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:10:18.320425 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de6ff0-08db-47c0-b295-701c33b2908c None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:10:18.320965 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de6ff0-08db-47c0-b295-701c33b2908c None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:10:18.321155 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de6ff0-08db-47c0-b295-701c33b2908c None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:10:18.322907 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 749/1476] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:10:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:10:18.328106 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7ef17-739b-441b-9a9f-09973fbca2b6 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:10:18.328536 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7ef17-739b-441b-9a9f-09973fbca2b6 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:10:18.328676 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7ef17-739b-441b-9a9f-09973fbca2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:10:18.346728 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-beb7ef17-739b-441b-9a9f-09973fbca2b6 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:10:18.347963 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-beb7ef17-739b-441b-9a9f-09973fbca2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:10:18.355790 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 728/1477] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:10:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:10:18.379023 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de6ff0-08db-47c0-b295-701c33b2908c None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:10:18.391748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de6ff0-08db-47c0-b295-701c33b2908c None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:10:18.396134 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32d037-80ce-4756-ba3e-14b2fea558b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:10:18.396771 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32d037-80ce-4756-ba3e-14b2fea558b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:10:18.397055 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32d037-80ce-4756-ba3e-14b2fea558b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:10:18.397331 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32d037-80ce-4756-ba3e-14b2fea558b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:10:18.398064 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc32d037-80ce-4756-ba3e-14b2fea558b9 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:10:18.400504 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: WARNING py.warnings [None req-cc32d037-80ce-4756-ba3e-14b2fea558b9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 03 16:10:18.400504 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: warnings.warn(msg) Jun 03 16:10:18.400504 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: Jun 03 16:10:18.401639 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc32d037-80ce-4756-ba3e-14b2fea558b9 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:10:18.420835 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 750/1478] 10.222.0.41 () {56 vars in 1108 bytes} [Fri Jun 3 16:10:18 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:10:20.652960 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03575633-2a7b-4f7e-87cb-caa9d50a6ef9 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:10:20.653499 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03575633-2a7b-4f7e-87cb-caa9d50a6ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:10:20.653499 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03575633-2a7b-4f7e-87cb-caa9d50a6ef9 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:10:20.654884 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 729/1479] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:10:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:10:20.660306 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e674981-ee56-4186-9b06-bae6b2689929 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:10:20.660705 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e674981-ee56-4186-9b06-bae6b2689929 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:10:20.660867 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e674981-ee56-4186-9b06-bae6b2689929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:10:20.679678 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-8e674981-ee56-4186-9b06-bae6b2689929 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:10:20.681532 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-8e674981-ee56-4186-9b06-bae6b2689929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:10:20.689550 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 751/1480] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:10:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:10:20.733268 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03575633-2a7b-4f7e-87cb-caa9d50a6ef9 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:10:20.747763 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03575633-2a7b-4f7e-87cb-caa9d50a6ef9 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:10:20.753242 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100b8f41-41b0-4b1f-bdd2-8c3ef7ac4166 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:10:20.753877 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100b8f41-41b0-4b1f-bdd2-8c3ef7ac4166 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:10:20.754162 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100b8f41-41b0-4b1f-bdd2-8c3ef7ac4166 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:10:20.754442 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100b8f41-41b0-4b1f-bdd2-8c3ef7ac4166 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:10:20.762580 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100b8f41-41b0-4b1f-bdd2-8c3ef7ac4166 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:10:20.764196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100b8f41-41b0-4b1f-bdd2-8c3ef7ac4166 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:10:20.773440 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 730/1481] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:10:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:10:23.251114 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753de59-1ebf-4434-a7d1-5a3fba0421f3 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:10:23.251676 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753de59-1ebf-4434-a7d1-5a3fba0421f3 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:10:23.252244 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753de59-1ebf-4434-a7d1-5a3fba0421f3 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:10:23.253057 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 752/1482] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:10:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:10:23.258466 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc5703-0ee3-4d31-8dea-5d7148aef494 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:10:23.258871 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc5703-0ee3-4d31-8dea-5d7148aef494 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:10:23.259025 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc5703-0ee3-4d31-8dea-5d7148aef494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:10:23.276783 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-8cbc5703-0ee3-4d31-8dea-5d7148aef494 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:10:23.278244 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8cbc5703-0ee3-4d31-8dea-5d7148aef494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:10:23.285838 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 731/1483] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:10:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:10:23.322501 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753de59-1ebf-4434-a7d1-5a3fba0421f3 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:10:23.333049 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753de59-1ebf-4434-a7d1-5a3fba0421f3 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:10:23.336833 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c0c908-9b00-45c8-839a-7cdd6f36ee21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:10:23.338090 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0c908-9b00-45c8-839a-7cdd6f36ee21 service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:10:23.338380 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0c908-9b00-45c8-839a-7cdd6f36ee21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:10:23.338660 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0c908-9b00-45c8-839a-7cdd6f36ee21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:10:23.347608 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c0c908-9b00-45c8-839a-7cdd6f36ee21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:10:23.349026 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c0c908-9b00-45c8-839a-7cdd6f36ee21 service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:10:23.358670 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 753/1484] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:10:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:10:25.660600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec807b-83c5-4bd9-b0f1-a65664155e01 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:10:25.661187 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec807b-83c5-4bd9-b0f1-a65664155e01 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:10:25.661187 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec807b-83c5-4bd9-b0f1-a65664155e01 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:10:25.662580 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 732/1485] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:10:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:10:25.668315 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdab372-7e4e-481c-96cc-160f599e4294 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:10:25.668735 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdab372-7e4e-481c-96cc-160f599e4294 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:10:25.668882 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdab372-7e4e-481c-96cc-160f599e4294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:10:25.687946 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-2fdab372-7e4e-481c-96cc-160f599e4294 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:10:25.689130 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-2fdab372-7e4e-481c-96cc-160f599e4294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:10:25.696774 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 754/1486] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:10:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:10:25.734192 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec807b-83c5-4bd9-b0f1-a65664155e01 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:10:25.745534 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec807b-83c5-4bd9-b0f1-a65664155e01 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:10:25.749308 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a066642c-3553-4935-9261-6ce19a8487f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:10:25.749990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a066642c-3553-4935-9261-6ce19a8487f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:10:25.750275 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a066642c-3553-4935-9261-6ce19a8487f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:10:25.750549 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a066642c-3553-4935-9261-6ce19a8487f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:10:25.759636 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a066642c-3553-4935-9261-6ce19a8487f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:10:25.761063 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a066642c-3553-4935-9261-6ce19a8487f2 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:10:25.770470 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 733/1487] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:10:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:10:28.260781 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045805e-ae1a-40ad-8b15-dfe557a36428 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:10:28.261232 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045805e-ae1a-40ad-8b15-dfe557a36428 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:10:28.261379 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045805e-ae1a-40ad-8b15-dfe557a36428 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:10:28.262746 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 755/1488] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:10:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:10:28.267896 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b6613-4686-4da7-93ef-9544d9ca5e3d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:10:28.268317 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b6613-4686-4da7-93ef-9544d9ca5e3d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:10:28.268463 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b6613-4686-4da7-93ef-9544d9ca5e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:10:28.287154 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-075b6613-4686-4da7-93ef-9544d9ca5e3d None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:10:28.288379 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-075b6613-4686-4da7-93ef-9544d9ca5e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:10:28.296738 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 734/1489] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:10:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:10:28.336195 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045805e-ae1a-40ad-8b15-dfe557a36428 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:10:28.347122 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045805e-ae1a-40ad-8b15-dfe557a36428 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:10:28.350567 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44591099-391b-4db9-8ef4-93ae5638bcbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:10:28.351186 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44591099-391b-4db9-8ef4-93ae5638bcbe service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:10:28.351518 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44591099-391b-4db9-8ef4-93ae5638bcbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:10:28.351795 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44591099-391b-4db9-8ef4-93ae5638bcbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:10:28.361072 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44591099-391b-4db9-8ef4-93ae5638bcbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:10:28.362469 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44591099-391b-4db9-8ef4-93ae5638bcbe service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:10:28.371511 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 756/1490] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:10:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:10:30.851655 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3a8ea-1ad5-46e5-8235-f22f28229b1e None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:10:30.852261 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3a8ea-1ad5-46e5-8235-f22f28229b1e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:10:30.852261 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3a8ea-1ad5-46e5-8235-f22f28229b1e None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:10:30.853616 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 735/1491] 10.222.0.41 () {54 vars in 926 bytes} [Fri Jun 3 16:10:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:10:30.858842 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ff577-6ebd-4902-87b4-598d5b1812c6 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:10:30.859240 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ff577-6ebd-4902-87b4-598d5b1812c6 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:10:30.859442 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ff577-6ebd-4902-87b4-598d5b1812c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:10:30.876883 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-ef2ff577-6ebd-4902-87b4-598d5b1812c6 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:10:30.878077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-ef2ff577-6ebd-4902-87b4-598d5b1812c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:10:30.886275 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 757/1492] 10.222.0.41 () {58 vars in 1023 bytes} [Fri Jun 3 16:10:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:10:30.925116 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da3a8ea-1ad5-46e5-8235-f22f28229b1e None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:10:30.935958 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da3a8ea-1ad5-46e5-8235-f22f28229b1e None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:10:30.939395 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6bac42-4b1c-446c-aead-38e421238124 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:10:30.940018 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6bac42-4b1c-446c-aead-38e421238124 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:10:30.940299 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6bac42-4b1c-446c-aead-38e421238124 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:10:30.940574 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6bac42-4b1c-446c-aead-38e421238124 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:10:30.949859 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6bac42-4b1c-446c-aead-38e421238124 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:10:30.951286 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6bac42-4b1c-446c-aead-38e421238124 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:10:30.960699 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 736/1493] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:10:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:10:40.871815 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=33 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:10:40.956518 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=33 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:11:56.489113 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef061932-0b7e-4d16-9426-03d6bd0055c7 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:11:56.489693 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef061932-0b7e-4d16-9426-03d6bd0055c7 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:11:56.489693 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef061932-0b7e-4d16-9426-03d6bd0055c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:11:56.507266 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-ef061932-0b7e-4d16-9426-03d6bd0055c7 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:11:56.508481 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-ef061932-0b7e-4d16-9426-03d6bd0055c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:11:56.516077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 758/1494] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:11:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:11:56.524286 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e624be-831a-49c7-ae50-f6b95328ca54 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:11:56.524753 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e624be-831a-49c7-ae50-f6b95328ca54 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:11:56.524863 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e624be-831a-49c7-ae50-f6b95328ca54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:11:56.542980 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c9e624be-831a-49c7-ae50-f6b95328ca54 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:11:56.544172 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c9e624be-831a-49c7-ae50-f6b95328ca54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:11:56.551961 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 737/1495] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:11:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:11:56.556387 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef061932-0b7e-4d16-9426-03d6bd0055c7 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:11:56.565935 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef061932-0b7e-4d16-9426-03d6bd0055c7 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:11:56.570471 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd43d45-10b7-496c-a122-ff705fcbcb26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:11:56.571408 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd43d45-10b7-496c-a122-ff705fcbcb26 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:11:56.571696 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd43d45-10b7-496c-a122-ff705fcbcb26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:11:56.571977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd43d45-10b7-496c-a122-ff705fcbcb26 admin admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:11:56.572734 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd43d45-10b7-496c-a122-ff705fcbcb26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:11:56.573048 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd43d45-10b7-496c-a122-ff705fcbcb26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:11:56.574064 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: WARNING py.warnings [None req-7fd43d45-10b7-496c-a122-ff705fcbcb26 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 03 16:11:56.574064 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: warnings.warn(msg) Jun 03 16:11:56.574064 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: Jun 03 16:11:56.574947 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd43d45-10b7-496c-a122-ff705fcbcb26 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:11:56.587031 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 759/1496] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:11:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:11:56.595150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e624be-831a-49c7-ae50-f6b95328ca54 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:11:56.604848 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e624be-831a-49c7-ae50-f6b95328ca54 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:11:56.610746 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1894225-662a-4b50-99f2-e49da44e298e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:11:56.611576 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1894225-662a-4b50-99f2-e49da44e298e admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:11:56.611868 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1894225-662a-4b50-99f2-e49da44e298e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:11:56.612241 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1894225-662a-4b50-99f2-e49da44e298e admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:11:56.613204 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1894225-662a-4b50-99f2-e49da44e298e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:11:56.615440 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: WARNING py.warnings [None req-c1894225-662a-4b50-99f2-e49da44e298e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 03 16:11:56.615440 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: warnings.warn(msg) Jun 03 16:11:56.615440 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: Jun 03 16:11:56.616627 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1894225-662a-4b50-99f2-e49da44e298e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:11:56.651810 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 738/1497] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:11:56 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:11:56.656267 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd43d45-10b7-496c-a122-ff705fcbcb26 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:11:56.666731 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd43d45-10b7-496c-a122-ff705fcbcb26 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:11:56.720565 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c80df-60aa-4ab2-b626-e8771c3c47e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:11:56.721298 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c80df-60aa-4ab2-b626-e8771c3c47e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:11:56.721584 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c80df-60aa-4ab2-b626-e8771c3c47e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:11:56.721857 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c80df-60aa-4ab2-b626-e8771c3c47e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:11:56.722534 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05c80df-60aa-4ab2-b626-e8771c3c47e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:11:56.723564 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: WARNING py.warnings [None req-f05c80df-60aa-4ab2-b626-e8771c3c47e5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 03 16:11:56.723564 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: warnings.warn(msg) Jun 03 16:11:56.723564 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: Jun 03 16:11:56.724744 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05c80df-60aa-4ab2-b626-e8771c3c47e5 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:11:56.744103 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 760/1498] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:11:56 2022] POST /identity/v3/users => generated 289 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:11:56.748028 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1894225-662a-4b50-99f2-e49da44e298e admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:11:56.758386 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1894225-662a-4b50-99f2-e49da44e298e admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:11:56.762825 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916ec53a-5906-4a6f-8560-e6ec5b73a073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:11:56.763511 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ec53a-5906-4a6f-8560-e6ec5b73a073 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:11:56.763797 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ec53a-5906-4a6f-8560-e6ec5b73a073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:11:56.764071 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ec53a-5906-4a6f-8560-e6ec5b73a073 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:11:56.764793 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916ec53a-5906-4a6f-8560-e6ec5b73a073 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:11:56.765801 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: WARNING py.warnings [None req-916ec53a-5906-4a6f-8560-e6ec5b73a073 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 03 16:11:56.765801 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: warnings.warn(msg) Jun 03 16:11:56.765801 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: Jun 03 16:11:56.766674 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916ec53a-5906-4a6f-8560-e6ec5b73a073 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:11:56.779942 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 739/1499] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:11:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:11:56.783834 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c80df-60aa-4ab2-b626-e8771c3c47e5 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:11:56.793533 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c80df-60aa-4ab2-b626-e8771c3c47e5 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:11:56.797979 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2935c679-c515-4a84-bc49-6a2c65d7a51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:11:56.798631 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935c679-c515-4a84-bc49-6a2c65d7a51c admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:11:56.798920 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935c679-c515-4a84-bc49-6a2c65d7a51c admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:11:56.799204 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935c679-c515-4a84-bc49-6a2c65d7a51c admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:11:56.799962 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2935c679-c515-4a84-bc49-6a2c65d7a51c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:11:56.800967 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: WARNING py.warnings [None req-2935c679-c515-4a84-bc49-6a2c65d7a51c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 03 16:11:56.800967 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: warnings.warn(msg) Jun 03 16:11:56.800967 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: Jun 03 16:11:56.801841 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2935c679-c515-4a84-bc49-6a2c65d7a51c admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:11:56.814876 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 761/1500] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:11:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:11:56.818725 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916ec53a-5906-4a6f-8560-e6ec5b73a073 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:11:56.828753 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916ec53a-5906-4a6f-8560-e6ec5b73a073 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:11:56.833494 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5732f-ddf7-484a-9902-0bd97dfde1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:11:56.834361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5732f-ddf7-484a-9902-0bd97dfde1a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:11:56.834646 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5732f-ddf7-484a-9902-0bd97dfde1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:11:56.834925 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5732f-ddf7-484a-9902-0bd97dfde1a7 admin admin] PATH_INFO: `/v3/projects/f1d87ed2c4d94431b696982ee7f2dc35/users/4d37a7f2803c489abd9027e496385730/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:11:56.849741 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b5732f-ddf7-484a-9902-0bd97dfde1a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1d87ed2c4d94431b696982ee7f2dc35, user_id=4d37a7f2803c489abd9027e496385730, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:11:56.850772 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: WARNING py.warnings [None req-41b5732f-ddf7-484a-9902-0bd97dfde1a7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 03 16:11:56.850772 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: warnings.warn(msg) Jun 03 16:11:56.850772 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: Jun 03 16:11:56.852991 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b5732f-ddf7-484a-9902-0bd97dfde1a7 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:11:56.866441 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 740/1501] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:11:56 2022] PUT /identity/v3/projects/f1d87ed2c4d94431b696982ee7f2dc35/users/4d37a7f2803c489abd9027e496385730/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:11:56.881355 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0e565-adf2-453f-b268-3af897ed788c None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:11:56.881671 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0e565-adf2-453f-b268-3af897ed788c None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:11:56.881947 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0e565-adf2-453f-b268-3af897ed788c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:11:56.909510 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-45f0e565-adf2-453f-b268-3af897ed788c None None] MFA Rules not processed for user `4d37a7f2803c489abd9027e496385730`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:11:56.910722 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-45f0e565-adf2-453f-b268-3af897ed788c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:11:56.973279 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 762/1502] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:11:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:11:56.986312 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a05b1d-8a8c-427d-a629-f72ecc4ad428 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:11:56.986602 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a05b1d-8a8c-427d-a629-f72ecc4ad428 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:11:56.986882 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a05b1d-8a8c-427d-a629-f72ecc4ad428 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:11:56.988279 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 741/1503] 10.222.0.41 () {54 vars in 963 bytes} [Fri Jun 3 16:11:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:11:56.993050 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23022a7-5603-49bd-bdd6-41fbefdd019a None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:11:56.993340 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23022a7-5603-49bd-bdd6-41fbefdd019a None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:11:56.993618 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23022a7-5603-49bd-bdd6-41fbefdd019a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:11:57.011355 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-f23022a7-5603-49bd-bdd6-41fbefdd019a None None] MFA Rules not processed for user `62a3bbb9720049b29eb37b0dd88bbad7`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:11:57.012472 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-f23022a7-5603-49bd-bdd6-41fbefdd019a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:11:57.065423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:11:57.074177 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 763/1504] 10.222.0.41 () {58 vars in 1060 bytes} [Fri Jun 3 16:11:56 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:11:57.081864 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a05b1d-8a8c-427d-a629-f72ecc4ad428 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:11:57.092826 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a05b1d-8a8c-427d-a629-f72ecc4ad428 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:11:57.096298 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7eeea-fa11-4e64-81d5-e70f34e0fa20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:11:57.096925 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7eeea-fa11-4e64-81d5-e70f34e0fa20 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:11:57.097203 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7eeea-fa11-4e64-81d5-e70f34e0fa20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:11:57.097473 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7eeea-fa11-4e64-81d5-e70f34e0fa20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:11:57.105436 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd7eeea-fa11-4e64-81d5-e70f34e0fa20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:11:57.106835 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd7eeea-fa11-4e64-81d5-e70f34e0fa20 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:11:57.115262 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 742/1505] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:11:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 16:11:57.156802 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb339ba-c3fc-4fd8-943c-ed582615e105 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:11:57.157092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb339ba-c3fc-4fd8-943c-ed582615e105 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:11:57.157369 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb339ba-c3fc-4fd8-943c-ed582615e105 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:11:57.158633 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 764/1506] 10.222.0.41 () {52 vars in 829 bytes} [Fri Jun 3 16:11:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:11:57.162311 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7eeea-fa11-4e64-81d5-e70f34e0fa20 service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:11:57.223304 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7eeea-fa11-4e64-81d5-e70f34e0fa20 service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:11:57.227934 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32cd59-c480-48cd-addb-62f3b077dec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:11:57.228805 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32cd59-c480-48cd-addb-62f3b077dec0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:11:57.229087 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32cd59-c480-48cd-addb-62f3b077dec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:11:57.229363 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32cd59-c480-48cd-addb-62f3b077dec0 admin admin] PATH_INFO: `/v3/users/4d37a7f2803c489abd9027e496385730` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:11:57.230541 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c32cd59-c480-48cd-addb-62f3b077dec0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d37a7f2803c489abd9027e496385730)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:11:57.231596 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: WARNING py.warnings [None req-5c32cd59-c480-48cd-addb-62f3b077dec0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 03 16:11:57.231596 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: warnings.warn(msg) Jun 03 16:11:57.231596 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: Jun 03 16:11:57.232792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c32cd59-c480-48cd-addb-62f3b077dec0 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:11:57.297637 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5c32cd59-c480-48cd-addb-62f3b077dec0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d37a7f2803c489abd9027e496385730'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:11:57.307098 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5c32cd59-c480-48cd-addb-62f3b077dec0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d37a7f2803c489abd9027e496385730'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:11:57.325283 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5c32cd59-c480-48cd-addb-62f3b077dec0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d37a7f2803c489abd9027e496385730'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:11:57.325953 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5c32cd59-c480-48cd-addb-62f3b077dec0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d37a7f2803c489abd9027e496385730'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:11:57.333918 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:11:57.339581 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5c32cd59-c480-48cd-addb-62f3b077dec0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d37a7f2803c489abd9027e496385730'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:11:57.346372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 743/1507] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:11:57 2022] DELETE /identity/v3/users/4d37a7f2803c489abd9027e496385730 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:11:57.350757 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e247b79-7c8b-4b68-98be-28fcbcbe5ae2 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:11:57.351050 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e247b79-7c8b-4b68-98be-28fcbcbe5ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:11:57.351362 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e247b79-7c8b-4b68-98be-28fcbcbe5ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:11:57.368737 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-6e247b79-7c8b-4b68-98be-28fcbcbe5ae2 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:11:57.369915 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-6e247b79-7c8b-4b68-98be-28fcbcbe5ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:11:57.426513 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 765/1508] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:11:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:11:57.437089 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32cd59-c480-48cd-addb-62f3b077dec0 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:11:57.438474 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5c32cd59-c480-48cd-addb-62f3b077dec0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:11:57.493758 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32cd59-c480-48cd-addb-62f3b077dec0 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:11:57.497337 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3d54c-c7cb-4de7-939c-af63c0714b41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:11:57.497988 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3d54c-c7cb-4de7-939c-af63c0714b41 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:11:57.498258 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3d54c-c7cb-4de7-939c-af63c0714b41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:11:57.498535 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3d54c-c7cb-4de7-939c-af63c0714b41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:11:57.507349 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a3d54c-c7cb-4de7-939c-af63c0714b41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:11:57.509518 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a3d54c-c7cb-4de7-939c-af63c0714b41 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:11:57.518717 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 744/1509] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:11:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 16:11:57.836775 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e247b79-7c8b-4b68-98be-28fcbcbe5ae2 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:11:57.838233 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-6e247b79-7c8b-4b68-98be-28fcbcbe5ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:11:57.849995 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e247b79-7c8b-4b68-98be-28fcbcbe5ae2 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:11:57.855043 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e745b23-07fe-4000-9b7f-ee46da9da648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:11:57.856115 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e745b23-07fe-4000-9b7f-ee46da9da648 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:11:57.856281 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e745b23-07fe-4000-9b7f-ee46da9da648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:11:57.856563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e745b23-07fe-4000-9b7f-ee46da9da648 admin admin] PATH_INFO: `/v3/projects/f1d87ed2c4d94431b696982ee7f2dc35` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:11:57.857810 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e745b23-07fe-4000-9b7f-ee46da9da648 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1d87ed2c4d94431b696982ee7f2dc35)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:11:57.858777 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: WARNING py.warnings [None req-0e745b23-07fe-4000-9b7f-ee46da9da648 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 03 16:11:57.858777 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: warnings.warn(msg) Jun 03 16:11:57.858777 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: Jun 03 16:11:57.860000 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e745b23-07fe-4000-9b7f-ee46da9da648 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:11:57.922051 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0e745b23-07fe-4000-9b7f-ee46da9da648 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1d87ed2c4d94431b696982ee7f2dc35', 'request_id': 'req-0e745b23-07fe-4000-9b7f-ee46da9da648'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:11:57.935351 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0e745b23-07fe-4000-9b7f-ee46da9da648 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1d87ed2c4d94431b696982ee7f2dc35', 'request_id': 'req-0e745b23-07fe-4000-9b7f-ee46da9da648'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:11:57.944439 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0e745b23-07fe-4000-9b7f-ee46da9da648 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1d87ed2c4d94431b696982ee7f2dc35', 'request_id': 'req-0e745b23-07fe-4000-9b7f-ee46da9da648'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:11:57.955086 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0e745b23-07fe-4000-9b7f-ee46da9da648 admin admin] The token cache is being invalidate because project f1d87ed2c4d94431b696982ee7f2dc35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:11:57.955410 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0e745b23-07fe-4000-9b7f-ee46da9da648 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:11:57.956131 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0e745b23-07fe-4000-9b7f-ee46da9da648 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:11:57.957663 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 766/1510] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:11:57 2022] DELETE /identity/v3/projects/f1d87ed2c4d94431b696982ee7f2dc35 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:12:07.514696 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=29 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:12:07.952218 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:13:38.001464 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894a929-ff60-4990-b28f-22cb3bdddb90 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:13:38.002076 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894a929-ff60-4990-b28f-22cb3bdddb90 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:13:38.002076 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894a929-ff60-4990-b28f-22cb3bdddb90 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:13:38.003441 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 745/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 3 16:13:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:13:38.008409 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11038e-f515-4e20-bc7b-bf0524bea549 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:13:38.008813 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11038e-f515-4e20-bc7b-bf0524bea549 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:13:38.008979 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11038e-f515-4e20-bc7b-bf0524bea549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:13:38.027758 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-8b11038e-f515-4e20-bc7b-bf0524bea549 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:13:38.028986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-8b11038e-f515-4e20-bc7b-bf0524bea549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:13:38.088003 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 767/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 3 16:13:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:13:38.258048 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7894a929-ff60-4990-b28f-22cb3bdddb90 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:13:38.259751 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7894a929-ff60-4990-b28f-22cb3bdddb90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:13:38.333037 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7894a929-ff60-4990-b28f-22cb3bdddb90 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:13:38.338356 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b676cf-f2e7-46e7-b85b-dff130fd0bd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:13:38.338987 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b676cf-f2e7-46e7-b85b-dff130fd0bd3 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:13:38.339277 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b676cf-f2e7-46e7-b85b-dff130fd0bd3 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:13:38.339579 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b676cf-f2e7-46e7-b85b-dff130fd0bd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:13:38.347927 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b676cf-f2e7-46e7-b85b-dff130fd0bd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:13:38.349487 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b676cf-f2e7-46e7-b85b-dff130fd0bd3 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:13:38.359536 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 746/1513] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:13:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:13:39.232172 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9d716-b3bb-4ab5-b349-eeeac8eda21a None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:13:39.232692 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9d716-b3bb-4ab5-b349-eeeac8eda21a None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:13:39.232692 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9d716-b3bb-4ab5-b349-eeeac8eda21a None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:13:39.234611 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 768/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 3 16:13:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:13:39.239496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf6c8b-3e9b-4be6-8a6b-0d8a49379db2 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:13:39.239897 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf6c8b-3e9b-4be6-8a6b-0d8a49379db2 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:13:39.240049 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf6c8b-3e9b-4be6-8a6b-0d8a49379db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:13:39.258562 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-5ecf6c8b-3e9b-4be6-8a6b-0d8a49379db2 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:13:39.259912 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5ecf6c8b-3e9b-4be6-8a6b-0d8a49379db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:13:39.268149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 747/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 3 16:13:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:13:39.421003 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9d716-b3bb-4ab5-b349-eeeac8eda21a None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:13:39.435238 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9d716-b3bb-4ab5-b349-eeeac8eda21a None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:13:39.440630 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd67310-1da1-4924-8679-bcc07ad2aad1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:13:39.441261 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd67310-1da1-4924-8679-bcc07ad2aad1 service nova] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:13:39.441544 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd67310-1da1-4924-8679-bcc07ad2aad1 service nova] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:13:39.441817 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd67310-1da1-4924-8679-bcc07ad2aad1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:13:39.449821 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd67310-1da1-4924-8679-bcc07ad2aad1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:13:39.451941 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd67310-1da1-4924-8679-bcc07ad2aad1 service nova] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:13:39.461242 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 769/1516] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:13:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:13:40.800042 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f2bc2-b221-41ad-9732-1b152a8add1f None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:13:40.800612 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f2bc2-b221-41ad-9732-1b152a8add1f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:13:40.800612 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f2bc2-b221-41ad-9732-1b152a8add1f None None] PATH_INFO: `/v3` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:13:40.801890 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 748/1517] 10.222.0.118 () {54 vars in 926 bytes} [Fri Jun 3 16:13:40 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:13:40.808896 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533707c6-45f2-4892-939b-65cc6c2d9a45 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:13:40.809292 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533707c6-45f2-4892-939b-65cc6c2d9a45 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:13:40.809459 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533707c6-45f2-4892-939b-65cc6c2d9a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:13:40.827573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-533707c6-45f2-4892-939b-65cc6c2d9a45 None None] MFA Rules not processed for user `be901786f8934d559e0bac5d622f97cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:13:40.828763 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-533707c6-45f2-4892-939b-65cc6c2d9a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:13:40.890151 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 770/1518] 10.222.0.118 () {58 vars in 1016 bytes} [Fri Jun 3 16:13:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:13:44.906752 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97682407-bf10-4907-97d8-06490aad7f63 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:13:44.907354 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97682407-bf10-4907-97d8-06490aad7f63 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:13:44.907354 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97682407-bf10-4907-97d8-06490aad7f63 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:13:44.908789 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 749/1519] 10.222.0.41 () {54 vars in 960 bytes} [Fri Jun 3 16:13:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:13:44.913770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906759b2-30f6-40c7-bb51-c6b6457e88d1 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:13:44.914208 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906759b2-30f6-40c7-bb51-c6b6457e88d1 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:13:44.914323 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906759b2-30f6-40c7-bb51-c6b6457e88d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:13:44.932019 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-906759b2-30f6-40c7-bb51-c6b6457e88d1 None None] MFA Rules not processed for user `be901786f8934d559e0bac5d622f97cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:13:44.933203 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-906759b2-30f6-40c7-bb51-c6b6457e88d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:13:44.941981 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 771/1520] 10.222.0.41 () {58 vars in 1057 bytes} [Fri Jun 3 16:13:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:13:44.949303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97682407-bf10-4907-97d8-06490aad7f63 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:13:44.963586 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97682407-bf10-4907-97d8-06490aad7f63 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:13:44.968427 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad928094-c8a3-4562-9e40-e6fcc7efa295 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be901786f8934d559e0bac5d622f97cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:13:44.969074 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad928094-c8a3-4562-9e40-e6fcc7efa295 service placement] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:13:44.969354 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad928094-c8a3-4562-9e40-e6fcc7efa295 service placement] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:13:44.969650 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad928094-c8a3-4562-9e40-e6fcc7efa295 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:13:44.978315 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad928094-c8a3-4562-9e40-e6fcc7efa295 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:13:44.980186 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad928094-c8a3-4562-9e40-e6fcc7efa295 service placement] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:13:44.989498 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 750/1521] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:13:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:13:50.742721 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b2caf-790a-4639-82ee-e8fbb1d9e2d4 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:13:50.743365 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b2caf-790a-4639-82ee-e8fbb1d9e2d4 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:13:50.743365 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b2caf-790a-4639-82ee-e8fbb1d9e2d4 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:13:50.744722 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 772/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 3 16:13:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:13:50.750320 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7204ff-432d-4a07-97b7-a61d535eff9c None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:13:50.750761 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7204ff-432d-4a07-97b7-a61d535eff9c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:13:50.750871 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7204ff-432d-4a07-97b7-a61d535eff9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:13:50.768367 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-7a7204ff-432d-4a07-97b7-a61d535eff9c None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:13:50.769562 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7a7204ff-432d-4a07-97b7-a61d535eff9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:13:50.777313 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 751/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 3 16:13:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:13:50.930207 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b2caf-790a-4639-82ee-e8fbb1d9e2d4 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:13:50.944714 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b2caf-790a-4639-82ee-e8fbb1d9e2d4 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:13:50.950145 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0c65f-b045-424f-99a9-9bc871da5c0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:13:50.950807 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0c65f-b045-424f-99a9-9bc871da5c0b service nova] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:13:50.951085 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0c65f-b045-424f-99a9-9bc871da5c0b service nova] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:13:50.951397 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0c65f-b045-424f-99a9-9bc871da5c0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:13:50.959773 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef0c65f-b045-424f-99a9-9bc871da5c0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:13:50.962107 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef0c65f-b045-424f-99a9-9bc871da5c0b service nova] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:13:50.972278 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 773/1524] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:13:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:13:51.830361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311e61b-0d4b-49de-ad23-0f305b21df40 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:13:51.830873 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311e61b-0d4b-49de-ad23-0f305b21df40 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:13:51.830873 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311e61b-0d4b-49de-ad23-0f305b21df40 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:13:51.832253 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 752/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 3 16:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:13:51.837016 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a99e23-a534-47d6-9797-fd9bbf83791d None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:13:51.837436 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a99e23-a534-47d6-9797-fd9bbf83791d None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:13:51.837578 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a99e23-a534-47d6-9797-fd9bbf83791d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:13:51.855185 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-e2a99e23-a534-47d6-9797-fd9bbf83791d None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:13:51.856423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-e2a99e23-a534-47d6-9797-fd9bbf83791d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:13:51.864261 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 774/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 3 16:13:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:13:52.012818 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e311e61b-0d4b-49de-ad23-0f305b21df40 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:13:52.027487 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e311e61b-0d4b-49de-ad23-0f305b21df40 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:13:52.033187 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61b53ac-aa40-4448-b9f2-daa4be4da6f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:13:52.033824 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b53ac-aa40-4448-b9f2-daa4be4da6f2 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:13:52.034113 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b53ac-aa40-4448-b9f2-daa4be4da6f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:13:52.034394 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b53ac-aa40-4448-b9f2-daa4be4da6f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:13:52.042399 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61b53ac-aa40-4448-b9f2-daa4be4da6f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:13:52.044038 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61b53ac-aa40-4448-b9f2-daa4be4da6f2 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:13:52.054610 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 753/1527] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:13:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:13:56.052383 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7ea29-111a-4c7f-b9ae-c426169eeb15 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:13:56.052925 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7ea29-111a-4c7f-b9ae-c426169eeb15 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:13:56.052925 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7ea29-111a-4c7f-b9ae-c426169eeb15 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:13:56.054242 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 775/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 16:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:13:56.059635 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86fa4d1-ed77-4ad9-aadc-a64a25fbe59b None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:13:56.060066 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86fa4d1-ed77-4ad9-aadc-a64a25fbe59b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:13:56.060196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86fa4d1-ed77-4ad9-aadc-a64a25fbe59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:13:56.078314 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c86fa4d1-ed77-4ad9-aadc-a64a25fbe59b None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:13:56.079546 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c86fa4d1-ed77-4ad9-aadc-a64a25fbe59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:13:56.087243 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 754/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 16:13:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:13:56.124893 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f7ea29-111a-4c7f-b9ae-c426169eeb15 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:13:56.126648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-29f7ea29-111a-4c7f-b9ae-c426169eeb15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:13:56.183223 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f7ea29-111a-4c7f-b9ae-c426169eeb15 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:13:56.187265 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcf6ea0-9818-49e2-83d4-b56222fbb4bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:13:56.187953 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf6ea0-9818-49e2-83d4-b56222fbb4bd service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:13:56.188237 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf6ea0-9818-49e2-83d4-b56222fbb4bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:13:56.188511 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf6ea0-9818-49e2-83d4-b56222fbb4bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:13:56.197480 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcf6ea0-9818-49e2-83d4-b56222fbb4bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:13:56.198961 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcf6ea0-9818-49e2-83d4-b56222fbb4bd service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:13:56.204452 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:13:56.209543 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 776/1530] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:13:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:13:58.426956 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf56145-1108-4b42-8393-cdb932d4db37 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:13:58.427509 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf56145-1108-4b42-8393-cdb932d4db37 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:13:58.427599 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf56145-1108-4b42-8393-cdb932d4db37 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:13:58.428900 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 755/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 16:13:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:13:58.434048 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf44b0-456e-416f-8bfe-5c1103137520 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:13:58.434485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf44b0-456e-416f-8bfe-5c1103137520 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:13:58.434625 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf44b0-456e-416f-8bfe-5c1103137520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:13:58.453591 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-aadf44b0-456e-416f-8bfe-5c1103137520 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:13:58.454793 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-aadf44b0-456e-416f-8bfe-5c1103137520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:13:58.462554 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 777/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 16:13:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:13:58.497370 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf56145-1108-4b42-8393-cdb932d4db37 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:13:58.508390 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf56145-1108-4b42-8393-cdb932d4db37 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:13:58.511776 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc468f-ede6-421d-978a-d3ec40a93d97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:13:58.512397 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc468f-ede6-421d-978a-d3ec40a93d97 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:13:58.512678 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc468f-ede6-421d-978a-d3ec40a93d97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:13:58.512952 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc468f-ede6-421d-978a-d3ec40a93d97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:13:58.522239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc468f-ede6-421d-978a-d3ec40a93d97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:13:58.523696 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc468f-ede6-421d-978a-d3ec40a93d97 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:13:58.532981 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 756/1533] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:13:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:14:02.294468 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fa11d-b750-4e93-94fe-be16fda51cbc None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:14:02.294912 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fa11d-b750-4e93-94fe-be16fda51cbc None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:14:02.295006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fa11d-b750-4e93-94fe-be16fda51cbc None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:14:02.296907 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 778/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 16:14:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:14:02.302226 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80daafb4-3b76-482a-bea2-aeb4e07c64e0 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:14:02.302650 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80daafb4-3b76-482a-bea2-aeb4e07c64e0 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:14:02.302787 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80daafb4-3b76-482a-bea2-aeb4e07c64e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:14:02.321300 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-80daafb4-3b76-482a-bea2-aeb4e07c64e0 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:14:02.322532 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-80daafb4-3b76-482a-bea2-aeb4e07c64e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:14:02.330328 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 757/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 16:14:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:14:02.598733 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4fa11d-b750-4e93-94fe-be16fda51cbc None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:14:02.600734 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-8f4fa11d-b750-4e93-94fe-be16fda51cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:14:02.658228 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4fa11d-b750-4e93-94fe-be16fda51cbc None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:14:02.661800 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e357798b-e228-4a87-bfba-411221ccc567 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:14:02.662445 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e357798b-e228-4a87-bfba-411221ccc567 service cinder] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:14:02.662735 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e357798b-e228-4a87-bfba-411221ccc567 service cinder] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:14:02.663020 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e357798b-e228-4a87-bfba-411221ccc567 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:14:02.672711 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e357798b-e228-4a87-bfba-411221ccc567 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:14:02.674143 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e357798b-e228-4a87-bfba-411221ccc567 service cinder] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:14:02.683459 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 779/1536] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:14:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:14:04.955233 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64f651-e46e-4051-87e6-ccfb74d57732 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:14:04.955923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64f651-e46e-4051-87e6-ccfb74d57732 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:14:04.955923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64f651-e46e-4051-87e6-ccfb74d57732 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:14:04.957252 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 758/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 16:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:14:04.962629 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03fbd1-e244-44e5-92d8-afa5bd0ed359 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:14:04.963127 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03fbd1-e244-44e5-92d8-afa5bd0ed359 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:14:04.963233 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03fbd1-e244-44e5-92d8-afa5bd0ed359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:14:04.981542 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-8a03fbd1-e244-44e5-92d8-afa5bd0ed359 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:14:04.982737 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-8a03fbd1-e244-44e5-92d8-afa5bd0ed359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:14:04.990369 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 780/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 16:14:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:14:05.259503 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64f651-e46e-4051-87e6-ccfb74d57732 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:14:05.261185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ad64f651-e46e-4051-87e6-ccfb74d57732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:14:05.273836 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64f651-e46e-4051-87e6-ccfb74d57732 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:14:05.277324 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f2137-4576-4329-8c6d-f94907e4c336 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:14:05.277950 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f2137-4576-4329-8c6d-f94907e4c336 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:14:05.278233 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f2137-4576-4329-8c6d-f94907e4c336 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:14:05.278508 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f2137-4576-4329-8c6d-f94907e4c336 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:14:05.287545 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8f2137-4576-4329-8c6d-f94907e4c336 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:14:05.288974 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8f2137-4576-4329-8c6d-f94907e4c336 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:14:05.298688 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 759/1539] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:14:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:14:14.976274 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=32 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:14:15.294037 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=70 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:35.099250 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5fc54e-474f-484f-a64c-cf449a3a9dc2 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:35.099863 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5fc54e-474f-484f-a64c-cf449a3a9dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:35.099863 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5fc54e-474f-484f-a64c-cf449a3a9dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:35.147662 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-9f5fc54e-474f-484f-a64c-cf449a3a9dc2 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:35.148920 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-9f5fc54e-474f-484f-a64c-cf449a3a9dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:35.159774 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 781/1540] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:35.168913 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be95e2-b000-49f4-a53e-f02a9fd21a9e None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:35.169949 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be95e2-b000-49f4-a53e-f02a9fd21a9e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:35.170446 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be95e2-b000-49f4-a53e-f02a9fd21a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:35.197787 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-28be95e2-b000-49f4-a53e-f02a9fd21a9e None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:35.199542 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-28be95e2-b000-49f4-a53e-f02a9fd21a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:35.209821 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 760/1541] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:35.215055 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5fc54e-474f-484f-a64c-cf449a3a9dc2 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:35.239504 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5fc54e-474f-484f-a64c-cf449a3a9dc2 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:35.256172 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd26b48-f873-4435-8ef2-9f96d352d10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:35.257047 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd26b48-f873-4435-8ef2-9f96d352d10c admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:35.257342 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd26b48-f873-4435-8ef2-9f96d352d10c admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:35.257637 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd26b48-f873-4435-8ef2-9f96d352d10c admin admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:35.258387 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd26b48-f873-4435-8ef2-9f96d352d10c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:15:35.258697 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd26b48-f873-4435-8ef2-9f96d352d10c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:35.260401 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd26b48-f873-4435-8ef2-9f96d352d10c admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:35.276981 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 782/1542] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:15:35.297432 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be95e2-b000-49f4-a53e-f02a9fd21a9e None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:35.310039 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be95e2-b000-49f4-a53e-f02a9fd21a9e None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:35.319036 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fce57aa-f5e2-4be3-9769-0a7e9610fc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:35.320281 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce57aa-f5e2-4be3-9769-0a7e9610fc43 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:35.320807 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce57aa-f5e2-4be3-9769-0a7e9610fc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:35.321305 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce57aa-f5e2-4be3-9769-0a7e9610fc43 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:35.322273 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fce57aa-f5e2-4be3-9769-0a7e9610fc43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:35.324518 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fce57aa-f5e2-4be3-9769-0a7e9610fc43 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:35.369055 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 761/1543] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:15:35 2022] POST /identity/v3/projects => generated 389 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:15:35.373804 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd26b48-f873-4435-8ef2-9f96d352d10c admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:35.388874 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd26b48-f873-4435-8ef2-9f96d352d10c admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:35.471714 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0059d-9676-457c-b311-f430e563d203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:35.472813 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0059d-9676-457c-b311-f430e563d203 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:35.472813 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0059d-9676-457c-b311-f430e563d203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:35.473040 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0059d-9676-457c-b311-f430e563d203 admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:35.473710 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d0059d-9676-457c-b311-f430e563d203 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:35.475687 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d0059d-9676-457c-b311-f430e563d203 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:35.515655 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19cb29-a13f-4b75-a11f-77b937df1784 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:35.516502 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19cb29-a13f-4b75-a11f-77b937df1784 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:35.517035 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19cb29-a13f-4b75-a11f-77b937df1784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:35.523099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 783/1544] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:15:35 2022] POST /identity/v3/users => generated 295 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:15:35.524183 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0059d-9676-457c-b311-f430e563d203 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:35.535136 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0059d-9676-457c-b311-f430e563d203 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:35.545767 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c65d3-4785-44a2-912d-133741af4dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:35.546448 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c65d3-4785-44a2-912d-133741af4dc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:35.546732 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c65d3-4785-44a2-912d-133741af4dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:35.547009 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c65d3-4785-44a2-912d-133741af4dc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:35.547808 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c65d3-4785-44a2-912d-133741af4dc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:35.549414 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c65d3-4785-44a2-912d-133741af4dc7 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:35.550039 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-2e19cb29-a13f-4b75-a11f-77b937df1784 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:35.555725 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2e19cb29-a13f-4b75-a11f-77b937df1784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:35.567232 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 762/1546] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:35.567437 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 784/1546] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:15:35.571522 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c65d3-4785-44a2-912d-133741af4dc7 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:35.580492 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbd4f6-37bf-4220-8251-216787697922 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:35.580781 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbd4f6-37bf-4220-8251-216787697922 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:35.581064 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbd4f6-37bf-4220-8251-216787697922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:35.596982 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c65d3-4785-44a2-912d-133741af4dc7 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:35.609398 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c8bbd4f6-37bf-4220-8251-216787697922 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:35.610601 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c8bbd4f6-37bf-4220-8251-216787697922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:35.611486 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199a6dc-0a4b-4c7d-92de-21c99af72beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:35.612163 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199a6dc-0a4b-4c7d-92de-21c99af72beb admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:35.612443 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199a6dc-0a4b-4c7d-92de-21c99af72beb admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:35.612716 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199a6dc-0a4b-4c7d-92de-21c99af72beb admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:35.613441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199a6dc-0a4b-4c7d-92de-21c99af72beb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:35.615251 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199a6dc-0a4b-4c7d-92de-21c99af72beb admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:35.639687 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 785/1547] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:15:35.641330 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 763/1548] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:35.646010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199a6dc-0a4b-4c7d-92de-21c99af72beb admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:35.648159 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bbd4f6-37bf-4220-8251-216787697922 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:35.663268 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bbd4f6-37bf-4220-8251-216787697922 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:35.672676 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199a6dc-0a4b-4c7d-92de-21c99af72beb admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:35.678422 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab6b6d-b4f1-4f49-a95f-9ba336287dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:35.679097 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab6b6d-b4f1-4f49-a95f-9ba336287dce admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:35.679403 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab6b6d-b4f1-4f49-a95f-9ba336287dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:35.679681 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab6b6d-b4f1-4f49-a95f-9ba336287dce admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:35.680422 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab6b6d-b4f1-4f49-a95f-9ba336287dce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:15:35.680733 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab6b6d-b4f1-4f49-a95f-9ba336287dce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:35.682336 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: WARNING py.warnings [None req-96ab6b6d-b4f1-4f49-a95f-9ba336287dce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 03 16:15:35.682336 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: warnings.warn(msg) Jun 03 16:15:35.682336 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: Jun 03 16:15:35.683220 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab6b6d-b4f1-4f49-a95f-9ba336287dce admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:35.686906 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef03f8-0683-4854-b767-2342ee7f0d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:35.687842 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef03f8-0683-4854-b767-2342ee7f0d17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:35.688365 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef03f8-0683-4854-b767-2342ee7f0d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:35.688648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef03f8-0683-4854-b767-2342ee7f0d17 admin admin] PATH_INFO: `/v3/projects/ffa464e4d2264df68897a6d4b8dc7aab/users/94f91ae071b24e31a8ef8d50933ba4fb/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:35.699758 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 764/1549] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:15:35.701432 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f2bc7-e44d-4451-be14-dab5f3ef0663 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:35.701940 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f2bc7-e44d-4451-be14-dab5f3ef0663 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:35.702436 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f2bc7-e44d-4451-be14-dab5f3ef0663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:35.724032 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef03f8-0683-4854-b767-2342ee7f0d17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffa464e4d2264df68897a6d4b8dc7aab, user_id=94f91ae071b24e31a8ef8d50933ba4fb, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:35.724803 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: WARNING py.warnings [None req-66ef03f8-0683-4854-b767-2342ee7f0d17 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 03 16:15:35.724803 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: warnings.warn(msg) Jun 03 16:15:35.724803 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: Jun 03 16:15:35.726715 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef03f8-0683-4854-b767-2342ee7f0d17 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:35.730425 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-370f2bc7-e44d-4451-be14-dab5f3ef0663 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:35.732460 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-370f2bc7-e44d-4451-be14-dab5f3ef0663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:35.747567 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 765/1550] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:35.748741 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f2bc7-e44d-4451-be14-dab5f3ef0663 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:35.758335 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 786/1551] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:15:35 2022] PUT /identity/v3/projects/ffa464e4d2264df68897a6d4b8dc7aab/users/94f91ae071b24e31a8ef8d50933ba4fb/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:15:35.765638 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7db4e-ee35-41c3-9f13-3a61bd2aff87 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:35.765939 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7db4e-ee35-41c3-9f13-3a61bd2aff87 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:35.766225 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7db4e-ee35-41c3-9f13-3a61bd2aff87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:35.766550 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f2bc7-e44d-4451-be14-dab5f3ef0663 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:35.809900 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-b0a7db4e-ee35-41c3-9f13-3a61bd2aff87 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:35.811113 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-b0a7db4e-ee35-41c3-9f13-3a61bd2aff87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:35.835394 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2551d143-646a-42c4-91db-44c9595fd836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:35.836536 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551d143-646a-42c4-91db-44c9595fd836 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:35.837299 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551d143-646a-42c4-91db-44c9595fd836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:35.838072 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551d143-646a-42c4-91db-44c9595fd836 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:35.839689 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2551d143-646a-42c4-91db-44c9595fd836 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:35.843107 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2551d143-646a-42c4-91db-44c9595fd836 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:35.886975 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 766/1552] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:15:35 2022] POST /identity/v3/projects => generated 385 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:15:35.888860 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da7f8d-f679-4975-87ef-97f48fea5bbc None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:35.889158 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da7f8d-f679-4975-87ef-97f48fea5bbc None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:35.889434 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da7f8d-f679-4975-87ef-97f48fea5bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:35.904936 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 787/1553] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:35.906234 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a0e31-5287-4e2b-8492-a977554570f0 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:35.906532 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a0e31-5287-4e2b-8492-a977554570f0 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:35.906810 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a0e31-5287-4e2b-8492-a977554570f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:35.914809 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-17da7f8d-f679-4975-87ef-97f48fea5bbc None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:35.916183 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-17da7f8d-f679-4975-87ef-97f48fea5bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:35.924323 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-310a0e31-5287-4e2b-8492-a977554570f0 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:35.925553 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-310a0e31-5287-4e2b-8492-a977554570f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:35.944238 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:35.977072 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 767/1554] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:35.978454 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ea274-7941-489b-9453-ba6c86bcb0e3 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:35.978747 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ea274-7941-489b-9453-ba6c86bcb0e3 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:35.979045 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ea274-7941-489b-9453-ba6c86bcb0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:35.991004 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 788/1555] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:35.992323 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076268a3-b22e-4487-bb40-dded5cb1805f None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:35.992616 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076268a3-b22e-4487-bb40-dded5cb1805f None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:35.992895 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076268a3-b22e-4487-bb40-dded5cb1805f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:35.997381 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-124ea274-7941-489b-9453-ba6c86bcb0e3 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:35.998587 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-124ea274-7941-489b-9453-ba6c86bcb0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:36.006466 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 768/1556] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:36.007797 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2a836-b95c-4b47-8dee-cedbe726564b None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:36.008085 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2a836-b95c-4b47-8dee-cedbe726564b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:36.008364 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2a836-b95c-4b47-8dee-cedbe726564b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:36.011025 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-076268a3-b22e-4487-bb40-dded5cb1805f None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:36.012329 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-076268a3-b22e-4487-bb40-dded5cb1805f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:36.020719 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 789/1557] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:36.022111 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076268a3-b22e-4487-bb40-dded5cb1805f None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:36.026881 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-24d2a836-b95c-4b47-8dee-cedbe726564b None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:36.028817 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-24d2a836-b95c-4b47-8dee-cedbe726564b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:36.033192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076268a3-b22e-4487-bb40-dded5cb1805f None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:36.036566 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 769/1558] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:36.037832 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7c280-1d53-488c-81d2-e08c21974398 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:36.038123 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7c280-1d53-488c-81d2-e08c21974398 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:36.038402 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7c280-1d53-488c-81d2-e08c21974398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:36.042957 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d2a42a-35a2-4413-84d4-246235316f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:36.043765 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2a42a-35a2-4413-84d4-246235316f05 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:36.044060 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2a42a-35a2-4413-84d4-246235316f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:36.044347 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2a42a-35a2-4413-84d4-246235316f05 admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:36.045353 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d2a42a-35a2-4413-84d4-246235316f05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:36.048316 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d2a42a-35a2-4413-84d4-246235316f05 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:36.056063 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c1c7c280-1d53-488c-81d2-e08c21974398 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:36.057334 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c1c7c280-1d53-488c-81d2-e08c21974398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:36.066839 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 770/1559] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:36.068243 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97793001-24f2-4d58-b28c-25b3a7eb50d9 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:36.068536 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97793001-24f2-4d58-b28c-25b3a7eb50d9 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:36.068815 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97793001-24f2-4d58-b28c-25b3a7eb50d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:36.072377 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 790/1560] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:15:36 2022] POST /identity/v3/users => generated 293 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:15:36.073389 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d2a42a-35a2-4413-84d4-246235316f05 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:36.077150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:36.088265 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d2a42a-35a2-4413-84d4-246235316f05 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:36.088635 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-97793001-24f2-4d58-b28c-25b3a7eb50d9 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:36.089833 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-97793001-24f2-4d58-b28c-25b3a7eb50d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:36.093630 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ede84c-988f-401e-a786-0cc46fae2a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:36.094319 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ede84c-988f-401e-a786-0cc46fae2a72 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:36.094602 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ede84c-988f-401e-a786-0cc46fae2a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:36.094881 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ede84c-988f-401e-a786-0cc46fae2a72 admin admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:36.095640 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ede84c-988f-401e-a786-0cc46fae2a72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:15:36.095956 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ede84c-988f-401e-a786-0cc46fae2a72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:36.097564 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ede84c-988f-401e-a786-0cc46fae2a72 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:36.098357 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 771/1561] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:36.099351 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97793001-24f2-4d58-b28c-25b3a7eb50d9 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:36.100694 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-97793001-24f2-4d58-b28c-25b3a7eb50d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:36.118752 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 791/1562] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:15:36.118752 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472433c-6fce-4a74-b0eb-9d031a6257ba None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:36.118752 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472433c-6fce-4a74-b0eb-9d031a6257ba None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:36.118752 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472433c-6fce-4a74-b0eb-9d031a6257ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:36.133149 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-0472433c-6fce-4a74-b0eb-9d031a6257ba None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:36.134392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-0472433c-6fce-4a74-b0eb-9d031a6257ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:36.142769 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 792/1563] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:36.144090 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc000f6f-81d3-4234-a428-0ebb05dc9f26 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:36.144389 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc000f6f-81d3-4234-a428-0ebb05dc9f26 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:36.144670 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc000f6f-81d3-4234-a428-0ebb05dc9f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:36.161508 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97793001-24f2-4d58-b28c-25b3a7eb50d9 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:36.163211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-bc000f6f-81d3-4234-a428-0ebb05dc9f26 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:36.164433 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-bc000f6f-81d3-4234-a428-0ebb05dc9f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:36.165239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05db2f8-e338-4a4d-8e60-c943a4b36428 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:36.165822 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05db2f8-e338-4a4d-8e60-c943a4b36428 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:36.166107 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05db2f8-e338-4a4d-8e60-c943a4b36428 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:36.166381 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05db2f8-e338-4a4d-8e60-c943a4b36428 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:36.172184 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 793/1564] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:36.173449 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3f890-cc48-42a3-90ef-e7c0793d11bd None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:36.173738 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3f890-cc48-42a3-90ef-e7c0793d11bd None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:36.174018 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3f890-cc48-42a3-90ef-e7c0793d11bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:36.178320 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05db2f8-e338-4a4d-8e60-c943a4b36428 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:36.179785 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05db2f8-e338-4a4d-8e60-c943a4b36428 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:36.189418 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 772/1565] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:15:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:15:36.190211 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05db2f8-e338-4a4d-8e60-c943a4b36428 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:36.191857 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-98f3f890-cc48-42a3-90ef-e7c0793d11bd None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:36.193045 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-98f3f890-cc48-42a3-90ef-e7c0793d11bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:36.199465 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05db2f8-e338-4a4d-8e60-c943a4b36428 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:36.201753 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 794/1566] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:36.202814 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669592a4-ff28-485c-9b6a-a8de07ba43b6 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:36.203099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669592a4-ff28-485c-9b6a-a8de07ba43b6 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:36.203418 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669592a4-ff28-485c-9b6a-a8de07ba43b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:36.208299 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b3e8a-4453-4d31-9a44-fdcd4abbbce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:36.209334 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b3e8a-4453-4d31-9a44-fdcd4abbbce9 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:36.209889 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b3e8a-4453-4d31-9a44-fdcd4abbbce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:36.210174 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b3e8a-4453-4d31-9a44-fdcd4abbbce9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:36.211189 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b3e8a-4453-4d31-9a44-fdcd4abbbce9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:36.213283 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b3e8a-4453-4d31-9a44-fdcd4abbbce9 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:36.222907 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-669592a4-ff28-485c-9b6a-a8de07ba43b6 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:36.224640 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-669592a4-ff28-485c-9b6a-a8de07ba43b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:36.232511 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 795/1567] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:36.232708 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 773/1568] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:15:36.233970 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b3e8a-4453-4d31-9a44-fdcd4abbbce9 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:36.236494 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669592a4-ff28-485c-9b6a-a8de07ba43b6 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:36.245121 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b3e8a-4453-4d31-9a44-fdcd4abbbce9 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:36.249703 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669592a4-ff28-485c-9b6a-a8de07ba43b6 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:36.255021 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393382c-5b60-4779-9401-2a72a8fcbee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:36.255725 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393382c-5b60-4779-9401-2a72a8fcbee2 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:36.256045 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393382c-5b60-4779-9401-2a72a8fcbee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:36.256343 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393382c-5b60-4779-9401-2a72a8fcbee2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:36.257026 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393382c-5b60-4779-9401-2a72a8fcbee2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:36.258078 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: WARNING py.warnings [None req-f393382c-5b60-4779-9401-2a72a8fcbee2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 03 16:15:36.258078 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: warnings.warn(msg) Jun 03 16:15:36.258078 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: Jun 03 16:15:36.259268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393382c-5b60-4779-9401-2a72a8fcbee2 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:36.261256 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0136f2e-b03e-4f35-8369-330d0d64a6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:36.263227 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0136f2e-b03e-4f35-8369-330d0d64a6d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:36.263963 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0136f2e-b03e-4f35-8369-330d0d64a6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:36.264688 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0136f2e-b03e-4f35-8369-330d0d64a6d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:36.266125 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0136f2e-b03e-4f35-8369-330d0d64a6d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:15:36.266592 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0136f2e-b03e-4f35-8369-330d0d64a6d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:36.268215 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0136f2e-b03e-4f35-8369-330d0d64a6d0 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:36.285748 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 774/1569] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:15:36.286999 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0136f2e-b03e-4f35-8369-330d0d64a6d0 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:36.295753 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 796/1570] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:15:36 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:15:36.296742 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393382c-5b60-4779-9401-2a72a8fcbee2 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:36.302282 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0136f2e-b03e-4f35-8369-330d0d64a6d0 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:36.367827 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de9251-8486-423c-8fd7-4817191d2b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:36.368516 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de9251-8486-423c-8fd7-4817191d2b8a admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:36.368648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393382c-5b60-4779-9401-2a72a8fcbee2 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:36.368826 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de9251-8486-423c-8fd7-4817191d2b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:36.369132 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de9251-8486-423c-8fd7-4817191d2b8a admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:36.369804 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de9251-8486-423c-8fd7-4817191d2b8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:15:36.370188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de9251-8486-423c-8fd7-4817191d2b8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:36.377644 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de9251-8486-423c-8fd7-4817191d2b8a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:36.383137 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b729db-b67f-4b17-9f5e-f8a1818086a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:36.383862 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b729db-b67f-4b17-9f5e-f8a1818086a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:36.384155 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b729db-b67f-4b17-9f5e-f8a1818086a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:36.384439 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b729db-b67f-4b17-9f5e-f8a1818086a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:36.385323 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b729db-b67f-4b17-9f5e-f8a1818086a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:15:36.385623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b729db-b67f-4b17-9f5e-f8a1818086a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:36.387249 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b729db-b67f-4b17-9f5e-f8a1818086a0 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:36.391559 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 775/1571] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:15:36.392482 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de9251-8486-423c-8fd7-4817191d2b8a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:36.408386 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 797/1572] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:15:36.408386 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b729db-b67f-4b17-9f5e-f8a1818086a0 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:36.412132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b729db-b67f-4b17-9f5e-f8a1818086a0 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:36.413927 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de9251-8486-423c-8fd7-4817191d2b8a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:36.417553 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532dec7d-51dc-4f30-8007-a81acc16cfcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:36.418266 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532dec7d-51dc-4f30-8007-a81acc16cfcf admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:36.418645 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532dec7d-51dc-4f30-8007-a81acc16cfcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:36.419131 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532dec7d-51dc-4f30-8007-a81acc16cfcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:36.419966 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532dec7d-51dc-4f30-8007-a81acc16cfcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:36.420068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13d6fc-c691-4f37-a99a-b81451fbd0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:36.420828 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13d6fc-c691-4f37-a99a-b81451fbd0f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:36.421364 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13d6fc-c691-4f37-a99a-b81451fbd0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:36.421699 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13d6fc-c691-4f37-a99a-b81451fbd0f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:36.421835 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532dec7d-51dc-4f30-8007-a81acc16cfcf admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:36.422427 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b13d6fc-c691-4f37-a99a-b81451fbd0f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:15:36.422759 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b13d6fc-c691-4f37-a99a-b81451fbd0f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:36.426014 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b13d6fc-c691-4f37-a99a-b81451fbd0f2 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:36.433689 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 798/1573] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:15:36.435031 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532dec7d-51dc-4f30-8007-a81acc16cfcf admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:36.449877 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 776/1574] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:15:36.449877 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13d6fc-c691-4f37-a99a-b81451fbd0f2 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:36.450886 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532dec7d-51dc-4f30-8007-a81acc16cfcf admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:36.454798 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc22070-1a13-48bf-944d-595face0bac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:36.455634 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc22070-1a13-48bf-944d-595face0bac9 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:36.455995 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc22070-1a13-48bf-944d-595face0bac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:36.456351 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc22070-1a13-48bf-944d-595face0bac9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:36.457173 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc22070-1a13-48bf-944d-595face0bac9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:15:36.457552 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc22070-1a13-48bf-944d-595face0bac9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:36.459243 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc22070-1a13-48bf-944d-595face0bac9 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:36.461276 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13d6fc-c691-4f37-a99a-b81451fbd0f2 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:36.468642 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596acc60-eb5d-4765-a896-fba1abd0b7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:36.469321 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596acc60-eb5d-4765-a896-fba1abd0b7d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:36.469611 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596acc60-eb5d-4765-a896-fba1abd0b7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:36.469891 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596acc60-eb5d-4765-a896-fba1abd0b7d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:36.470595 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596acc60-eb5d-4765-a896-fba1abd0b7d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:36.472633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596acc60-eb5d-4765-a896-fba1abd0b7d0 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:36.474210 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 799/1575] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:15:36.474978 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc22070-1a13-48bf-944d-595face0bac9 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:36.485317 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc22070-1a13-48bf-944d-595face0bac9 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:36.490544 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc579e32-10e2-4264-b98f-3c7d602a18af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:36.491344 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc579e32-10e2-4264-b98f-3c7d602a18af admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:36.491716 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc579e32-10e2-4264-b98f-3c7d602a18af admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:36.492244 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc579e32-10e2-4264-b98f-3c7d602a18af admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:36.493050 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc579e32-10e2-4264-b98f-3c7d602a18af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:36.495130 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc579e32-10e2-4264-b98f-3c7d602a18af admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:36.500258 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 777/1576] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:15:36 2022] POST /identity/v3/projects => generated 363 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:15:36.501354 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596acc60-eb5d-4765-a896-fba1abd0b7d0 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:36.507649 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:36.525597 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 800/1577] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:15:36 2022] POST /identity/v3/users => generated 288 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:15:36.525597 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc579e32-10e2-4264-b98f-3c7d602a18af admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:36.535372 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc579e32-10e2-4264-b98f-3c7d602a18af admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:36.551393 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:36.581734 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596acc60-eb5d-4765-a896-fba1abd0b7d0 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:36.587338 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a124a17-184f-4b47-8170-db85790a0ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:36.588040 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a124a17-184f-4b47-8170-db85790a0ac0 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:36.588324 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a124a17-184f-4b47-8170-db85790a0ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:36.588603 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a124a17-184f-4b47-8170-db85790a0ac0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:36.589949 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a124a17-184f-4b47-8170-db85790a0ac0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:36.592415 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a124a17-184f-4b47-8170-db85790a0ac0 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:36.608853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac8a4a-63e1-4e6f-b7be-bf57f61ce547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:36.609688 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac8a4a-63e1-4e6f-b7be-bf57f61ce547 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:36.609982 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac8a4a-63e1-4e6f-b7be-bf57f61ce547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:36.610314 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac8a4a-63e1-4e6f-b7be-bf57f61ce547 admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:36.611029 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cac8a4a-63e1-4e6f-b7be-bf57f61ce547 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:36.613257 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cac8a4a-63e1-4e6f-b7be-bf57f61ce547 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:36.630750 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 778/1578] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:15:36 2022] POST /identity/v3/projects => generated 377 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:15:36.631830 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a124a17-184f-4b47-8170-db85790a0ac0 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:36.661073 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 801/1579] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:15:36 2022] POST /identity/v3/projects => generated 393 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:15:36.662470 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac8a4a-63e1-4e6f-b7be-bf57f61ce547 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:36.702712 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a124a17-184f-4b47-8170-db85790a0ac0 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:36.753956 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac8a4a-63e1-4e6f-b7be-bf57f61ce547 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:36.758666 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5fa389-879a-4d40-b637-3fb3abcbac0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:36.759356 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5fa389-879a-4d40-b637-3fb3abcbac0b admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:36.759643 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5fa389-879a-4d40-b637-3fb3abcbac0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:36.759936 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5fa389-879a-4d40-b637-3fb3abcbac0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:36.760618 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5fa389-879a-4d40-b637-3fb3abcbac0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:36.763095 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5fa389-879a-4d40-b637-3fb3abcbac0b admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:36.770135 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5cde21-7685-49db-a70a-e1dfa0844a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:36.771255 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5cde21-7685-49db-a70a-e1dfa0844a71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:36.771573 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5cde21-7685-49db-a70a-e1dfa0844a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:36.771856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5cde21-7685-49db-a70a-e1dfa0844a71 admin admin] PATH_INFO: `/v3/projects/c6758d3ab63e4ace8d2d32a4b391d942/users/ef7c1e449b864b50b3c71c39f19968a0/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:36.789310 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5cde21-7685-49db-a70a-e1dfa0844a71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6758d3ab63e4ace8d2d32a4b391d942, user_id=ef7c1e449b864b50b3c71c39f19968a0, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:36.793775 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 802/1580] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:15:36 2022] POST /identity/v3/projects => generated 369 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:15:36.793932 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5cde21-7685-49db-a70a-e1dfa0844a71 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:36.794052 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5fa389-879a-4d40-b637-3fb3abcbac0b admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:36.806385 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 779/1581] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:15:36 2022] PUT /identity/v3/projects/c6758d3ab63e4ace8d2d32a4b391d942/users/ef7c1e449b864b50b3c71c39f19968a0/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:15:36.806958 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5cde21-7685-49db-a70a-e1dfa0844a71 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:36.860304 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:36.874129 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5cde21-7685-49db-a70a-e1dfa0844a71 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:36.876859 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5fa389-879a-4d40-b637-3fb3abcbac0b admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:36.880345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b696638-7e11-4f76-9c7b-a3aed037e642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:36.881061 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b696638-7e11-4f76-9c7b-a3aed037e642 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:36.881381 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b696638-7e11-4f76-9c7b-a3aed037e642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:36.881494 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0adc298-8671-4aab-95e4-c421b1878bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:36.881675 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b696638-7e11-4f76-9c7b-a3aed037e642 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:36.882233 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0adc298-8671-4aab-95e4-c421b1878bab admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:36.882406 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0adc298-8671-4aab-95e4-c421b1878bab admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:36.882623 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b696638-7e11-4f76-9c7b-a3aed037e642 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:36.882739 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0adc298-8671-4aab-95e4-c421b1878bab admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:36.883606 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0adc298-8671-4aab-95e4-c421b1878bab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:36.883985 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: WARNING py.warnings [None req-2b696638-7e11-4f76-9c7b-a3aed037e642 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 03 16:15:36.883985 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: warnings.warn(msg) Jun 03 16:15:36.883985 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: Jun 03 16:15:36.885255 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0adc298-8671-4aab-95e4-c421b1878bab admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:36.885525 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b696638-7e11-4f76-9c7b-a3aed037e642 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:36.890913 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:36.911457 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 780/1582] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:15:36 2022] POST /identity/v3/users => generated 282 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:15:36.913009 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b696638-7e11-4f76-9c7b-a3aed037e642 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:36.928668 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 803/1583] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:15:36 2022] POST /identity/v3/projects => generated 405 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:15:36.929017 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b696638-7e11-4f76-9c7b-a3aed037e642 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:36.930299 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0adc298-8671-4aab-95e4-c421b1878bab admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:37.021353 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73adf403-4b56-4ef9-80a0-ed8b61241393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:37.022080 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73adf403-4b56-4ef9-80a0-ed8b61241393 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:37.022343 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73adf403-4b56-4ef9-80a0-ed8b61241393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:37.022952 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73adf403-4b56-4ef9-80a0-ed8b61241393 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:37.023768 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73adf403-4b56-4ef9-80a0-ed8b61241393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:37.025682 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73adf403-4b56-4ef9-80a0-ed8b61241393 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:37.027891 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0adc298-8671-4aab-95e4-c421b1878bab admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:37.036662 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ea0ea-4218-432f-a391-9b59af748204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:37.037129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ea0ea-4218-432f-a391-9b59af748204 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:37.037470 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ea0ea-4218-432f-a391-9b59af748204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:37.037744 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ea0ea-4218-432f-a391-9b59af748204 admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:37.038542 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ea0ea-4218-432f-a391-9b59af748204 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:37.041552 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ea0ea-4218-432f-a391-9b59af748204 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:37.046154 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 781/1584] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:15:37.046154 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73adf403-4b56-4ef9-80a0-ed8b61241393 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:37.069264 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73adf403-4b56-4ef9-80a0-ed8b61241393 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:37.076454 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 804/1585] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:15:36 2022] POST /identity/v3/users => generated 289 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:15:37.077013 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ea0ea-4218-432f-a391-9b59af748204 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:37.086200 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba137d-a3d7-444e-9d17-b57cc3fd0b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:37.086816 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba137d-a3d7-444e-9d17-b57cc3fd0b58 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:37.087111 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba137d-a3d7-444e-9d17-b57cc3fd0b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:37.087413 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba137d-a3d7-444e-9d17-b57cc3fd0b58 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:37.088125 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ba137d-a3d7-444e-9d17-b57cc3fd0b58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:37.090349 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ba137d-a3d7-444e-9d17-b57cc3fd0b58 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:37.091487 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ea0ea-4218-432f-a391-9b59af748204 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:37.098505 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b355a-39b7-49d2-ba01-2e6cccfb76f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:37.099319 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b355a-39b7-49d2-ba01-2e6cccfb76f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:37.099606 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b355a-39b7-49d2-ba01-2e6cccfb76f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:37.099898 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b355a-39b7-49d2-ba01-2e6cccfb76f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:37.101087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b355a-39b7-49d2-ba01-2e6cccfb76f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:37.104185 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b355a-39b7-49d2-ba01-2e6cccfb76f3 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:37.120463 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 782/1586] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:15:37 2022] POST /identity/v3/users => generated 297 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:15:37.122641 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492f6b5-1d90-4244-8b52-a5f213fe71a0 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:37.123127 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492f6b5-1d90-4244-8b52-a5f213fe71a0 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:37.123629 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492f6b5-1d90-4244-8b52-a5f213fe71a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:37.129513 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 805/1587] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:15:37 2022] POST /identity/v3/users => generated 285 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:15:37.130465 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b355a-39b7-49d2-ba01-2e6cccfb76f3 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:37.142216 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b355a-39b7-49d2-ba01-2e6cccfb76f3 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:37.144497 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-e492f6b5-1d90-4244-8b52-a5f213fe71a0 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:37.145963 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e492f6b5-1d90-4244-8b52-a5f213fe71a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:37.147058 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53403b3e-5e80-43a8-ad31-2495b54c3336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:37.148874 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53403b3e-5e80-43a8-ad31-2495b54c3336 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:37.149162 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53403b3e-5e80-43a8-ad31-2495b54c3336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:37.149620 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53403b3e-5e80-43a8-ad31-2495b54c3336 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:37.150816 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53403b3e-5e80-43a8-ad31-2495b54c3336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:37.152677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53403b3e-5e80-43a8-ad31-2495b54c3336 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:37.156262 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 783/1588] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:37.157429 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e492f6b5-1d90-4244-8b52-a5f213fe71a0 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:37.167932 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e492f6b5-1d90-4244-8b52-a5f213fe71a0 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:37.172159 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 806/1589] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:15:37.173936 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d9b02-7c7a-4f9b-8573-86d2cfa18cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:37.174953 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d9b02-7c7a-4f9b-8573-86d2cfa18cc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:37.175471 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d9b02-7c7a-4f9b-8573-86d2cfa18cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:37.175946 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53403b3e-5e80-43a8-ad31-2495b54c3336 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:37.176166 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d9b02-7c7a-4f9b-8573-86d2cfa18cc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:37.177045 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902d9b02-7c7a-4f9b-8573-86d2cfa18cc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:37.179361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902d9b02-7c7a-4f9b-8573-86d2cfa18cc5 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:37.186568 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53403b3e-5e80-43a8-ad31-2495b54c3336 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:37.192302 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f8720-27e2-4e97-9196-c1afebd178c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:37.193845 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f8720-27e2-4e97-9196-c1afebd178c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:37.194279 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f8720-27e2-4e97-9196-c1afebd178c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:37.194655 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f8720-27e2-4e97-9196-c1afebd178c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:37.195838 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f8720-27e2-4e97-9196-c1afebd178c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:37.197515 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f8720-27e2-4e97-9196-c1afebd178c2 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:37.203966 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 784/1590] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:15:37 2022] POST /identity/v3/users => generated 303 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:15:37.205400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d9b02-7c7a-4f9b-8573-86d2cfa18cc5 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:37.209411 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 807/1591] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:15:37.210244 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f8720-27e2-4e97-9196-c1afebd178c2 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:37.219486 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d9b02-7c7a-4f9b-8573-86d2cfa18cc5 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:37.222400 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f8720-27e2-4e97-9196-c1afebd178c2 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:37.226058 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d21bd84-c7bf-4650-943e-3b6f5c34fc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:37.230229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21bd84-c7bf-4650-943e-3b6f5c34fc70 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:37.230909 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21bd84-c7bf-4650-943e-3b6f5c34fc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:37.231483 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21bd84-c7bf-4650-943e-3b6f5c34fc70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:37.232382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed828a7-a99b-439e-ad86-15d71d59cb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:37.233132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed828a7-a99b-439e-ad86-15d71d59cb56 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:37.233526 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed828a7-a99b-439e-ad86-15d71d59cb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:37.233932 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed828a7-a99b-439e-ad86-15d71d59cb56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:37.234521 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d21bd84-c7bf-4650-943e-3b6f5c34fc70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:37.234815 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed828a7-a99b-439e-ad86-15d71d59cb56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:37.236662 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed828a7-a99b-439e-ad86-15d71d59cb56 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:37.237053 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d21bd84-c7bf-4650-943e-3b6f5c34fc70 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:37.250428 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 785/1592] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:15:37.251816 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d21bd84-c7bf-4650-943e-3b6f5c34fc70 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:37.255395 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 808/1593] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:15:37.256326 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed828a7-a99b-439e-ad86-15d71d59cb56 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:37.265742 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d21bd84-c7bf-4650-943e-3b6f5c34fc70 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:37.277860 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e7b6dc-47e4-469b-8810-178d4b0d84ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:37.278997 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7b6dc-47e4-469b-8810-178d4b0d84ec admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:37.279289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7b6dc-47e4-469b-8810-178d4b0d84ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:37.279587 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7b6dc-47e4-469b-8810-178d4b0d84ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:37.280320 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e7b6dc-47e4-469b-8810-178d4b0d84ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:37.281945 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e7b6dc-47e4-469b-8810-178d4b0d84ec admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:37.298811 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 786/1594] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:15:37.300122 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e7b6dc-47e4-469b-8810-178d4b0d84ec admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:37.311522 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e7b6dc-47e4-469b-8810-178d4b0d84ec admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:37.316831 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc258ca3-72a8-48d3-9807-bf6f96e7ddb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:37.317572 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc258ca3-72a8-48d3-9807-bf6f96e7ddb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:37.317926 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc258ca3-72a8-48d3-9807-bf6f96e7ddb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:37.318368 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc258ca3-72a8-48d3-9807-bf6f96e7ddb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:37.319231 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc258ca3-72a8-48d3-9807-bf6f96e7ddb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:37.320959 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc258ca3-72a8-48d3-9807-bf6f96e7ddb5 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:37.337473 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 787/1595] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:15:37.338722 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc258ca3-72a8-48d3-9807-bf6f96e7ddb5 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:37.339937 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed828a7-a99b-439e-ad86-15d71d59cb56 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:37.344973 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f37062-8d1d-41c4-bb33-f0e535875170 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:37.345728 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f37062-8d1d-41c4-bb33-f0e535875170 service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:37.346220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f37062-8d1d-41c4-bb33-f0e535875170 service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:37.346583 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f37062-8d1d-41c4-bb33-f0e535875170 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:37.349661 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc258ca3-72a8-48d3-9807-bf6f96e7ddb5 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:37.354591 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a711e-6dae-4d9d-925e-552e643b7a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:37.355341 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a711e-6dae-4d9d-925e-552e643b7a6e admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:37.355700 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a711e-6dae-4d9d-925e-552e643b7a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:37.356789 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a711e-6dae-4d9d-925e-552e643b7a6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:37.357727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a711e-6dae-4d9d-925e-552e643b7a6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:37.358823 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f37062-8d1d-41c4-bb33-f0e535875170 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:37.359746 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a711e-6dae-4d9d-925e-552e643b7a6e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:37.360295 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f37062-8d1d-41c4-bb33-f0e535875170 service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:37.374133 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 788/1596] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:15:37.374133 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a711e-6dae-4d9d-925e-552e643b7a6e admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:37.374538 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 809/1597] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:15:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:15:37.375519 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f37062-8d1d-41c4-bb33-f0e535875170 service neutron] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:37.386202 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f37062-8d1d-41c4-bb33-f0e535875170 service neutron] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:37.391123 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b739b624-380c-4e55-9813-0eec8aa4d13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:37.392043 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739b624-380c-4e55-9813-0eec8aa4d13a admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:37.392159 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739b624-380c-4e55-9813-0eec8aa4d13a admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:37.392406 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739b624-380c-4e55-9813-0eec8aa4d13a admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:37.393460 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b739b624-380c-4e55-9813-0eec8aa4d13a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:37.396471 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b739b624-380c-4e55-9813-0eec8aa4d13a admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:37.400960 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a711e-6dae-4d9d-925e-552e643b7a6e admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:37.411568 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 810/1598] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:15:37.411568 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b739b624-380c-4e55-9813-0eec8aa4d13a admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:37.417005 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14fa409-007f-4bae-9c2a-8a72d48ba689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:37.421267 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fa409-007f-4bae-9c2a-8a72d48ba689 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:37.421573 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fa409-007f-4bae-9c2a-8a72d48ba689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:37.421860 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fa409-007f-4bae-9c2a-8a72d48ba689 admin admin] PATH_INFO: `/v3/projects/d0f59694966848829048aee648dddccc/users/c68189df1f9b4f9aacfa319381849697/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:37.422558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b739b624-380c-4e55-9813-0eec8aa4d13a admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:37.428442 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b66c378-7534-4cd1-9467-d13bd23338a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:37.430092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66c378-7534-4cd1-9467-d13bd23338a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:37.430389 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66c378-7534-4cd1-9467-d13bd23338a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:37.430678 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66c378-7534-4cd1-9467-d13bd23338a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:37.431525 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b66c378-7534-4cd1-9467-d13bd23338a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:37.433204 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b66c378-7534-4cd1-9467-d13bd23338a5 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:37.452226 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 811/1599] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:15:37.453340 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b66c378-7534-4cd1-9467-d13bd23338a5 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:37.461139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14fa409-007f-4bae-9c2a-8a72d48ba689 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0f59694966848829048aee648dddccc, user_id=c68189df1f9b4f9aacfa319381849697, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:37.463852 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14fa409-007f-4bae-9c2a-8a72d48ba689 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:37.464339 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b66c378-7534-4cd1-9467-d13bd23338a5 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:37.470537 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad321e1-da9a-4676-8025-37e63c5005ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:37.471373 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad321e1-da9a-4676-8025-37e63c5005ff admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:37.471720 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad321e1-da9a-4676-8025-37e63c5005ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:37.472775 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad321e1-da9a-4676-8025-37e63c5005ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:37.472903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad321e1-da9a-4676-8025-37e63c5005ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:37.474450 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad321e1-da9a-4676-8025-37e63c5005ff admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:37.476223 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:37.490715 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 789/1600] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:15:37 2022] PUT /identity/v3/projects/d0f59694966848829048aee648dddccc/users/c68189df1f9b4f9aacfa319381849697/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:15:37.493787 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14fa409-007f-4bae-9c2a-8a72d48ba689 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:37.500061 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 812/1601] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:15:37.500061 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad321e1-da9a-4676-8025-37e63c5005ff admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:37.512481 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad321e1-da9a-4676-8025-37e63c5005ff admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:37.558918 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:37.598213 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6269eb-285c-41d5-b55a-b0451cc7f77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:37.598968 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6269eb-285c-41d5-b55a-b0451cc7f77e admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:37.599257 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6269eb-285c-41d5-b55a-b0451cc7f77e admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:37.599574 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6269eb-285c-41d5-b55a-b0451cc7f77e admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:37.600419 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6269eb-285c-41d5-b55a-b0451cc7f77e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:37.603096 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6269eb-285c-41d5-b55a-b0451cc7f77e admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:37.618067 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 813/1602] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:15:37.619087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6269eb-285c-41d5-b55a-b0451cc7f77e admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:37.635525 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6269eb-285c-41d5-b55a-b0451cc7f77e admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:37.640829 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b2ce77-012f-41a6-8ec8-6f7e8c3a977f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:37.641731 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2ce77-012f-41a6-8ec8-6f7e8c3a977f admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:37.642066 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2ce77-012f-41a6-8ec8-6f7e8c3a977f admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:37.642285 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14fa409-007f-4bae-9c2a-8a72d48ba689 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:37.642397 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2ce77-012f-41a6-8ec8-6f7e8c3a977f admin admin] PATH_INFO: `/v3/projects/b1dde95632be44e29e77bfddc2ad84b5/users/34ca0974e6dd49bdb6a56ec577dad3cb/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:37.651254 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2f727-cd50-49b0-9182-adbded5e24de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:37.652746 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2f727-cd50-49b0-9182-adbded5e24de admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:37.653222 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2f727-cd50-49b0-9182-adbded5e24de admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:37.654014 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2f727-cd50-49b0-9182-adbded5e24de admin admin] PATH_INFO: `/v3/projects/3a0bf140eda04cedac1bba0d46246b0a/users/866c56c184f14dde80e90670dc3e758f/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:37.667273 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b2ce77-012f-41a6-8ec8-6f7e8c3a977f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1dde95632be44e29e77bfddc2ad84b5, user_id=34ca0974e6dd49bdb6a56ec577dad3cb, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:37.671155 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b2ce77-012f-41a6-8ec8-6f7e8c3a977f admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:37.683589 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a2f727-cd50-49b0-9182-adbded5e24de admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a0bf140eda04cedac1bba0d46246b0a, user_id=866c56c184f14dde80e90670dc3e758f, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:37.686814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a2f727-cd50-49b0-9182-adbded5e24de admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:37.698585 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 814/1603] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:15:37 2022] PUT /identity/v3/projects/b1dde95632be44e29e77bfddc2ad84b5/users/34ca0974e6dd49bdb6a56ec577dad3cb/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:15:37.699495 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b2ce77-012f-41a6-8ec8-6f7e8c3a977f admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:37.705192 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 790/1604] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:15:37 2022] PUT /identity/v3/projects/3a0bf140eda04cedac1bba0d46246b0a/users/866c56c184f14dde80e90670dc3e758f/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:15:37.716009 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba32251-c363-4559-b0e1-836552033f3e None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:37.716009 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba32251-c363-4559-b0e1-836552033f3e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:37.716009 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba32251-c363-4559-b0e1-836552033f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:37.728302 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-3ba32251-c363-4559-b0e1-836552033f3e None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:37.729629 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3ba32251-c363-4559-b0e1-836552033f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:37.836836 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 791/1605] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:37.838889 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba32251-c363-4559-b0e1-836552033f3e None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:37.860517 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba32251-c363-4559-b0e1-836552033f3e None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:37.875989 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b2ce77-012f-41a6-8ec8-6f7e8c3a977f admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:37.880342 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952dd0f2-12f9-425c-ba85-efb831bf0550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:37.881257 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952dd0f2-12f9-425c-ba85-efb831bf0550 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:37.881572 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952dd0f2-12f9-425c-ba85-efb831bf0550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:37.881803 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952dd0f2-12f9-425c-ba85-efb831bf0550 admin admin] PATH_INFO: `/v3/projects/34a04db02b7148da89f56b89dc1921eb/users/56b6a40596ce4c0fab0cf7def22fbc54/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:37.885789 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a0ce8-0344-4e45-a678-d58729ac1a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:37.887275 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a0ce8-0344-4e45-a678-d58729ac1a56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:37.887722 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a0ce8-0344-4e45-a678-d58729ac1a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:37.888170 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a0ce8-0344-4e45-a678-d58729ac1a56 admin admin] PATH_INFO: `/v3/projects/1443b802dc724950b5eb5c6446e2515c/users/4206a21c317e4b5aace8541699f8032d/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:37.908467 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40a0ce8-0344-4e45-a678-d58729ac1a56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1443b802dc724950b5eb5c6446e2515c, user_id=4206a21c317e4b5aace8541699f8032d, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:37.911953 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40a0ce8-0344-4e45-a678-d58729ac1a56 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:37.915223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952dd0f2-12f9-425c-ba85-efb831bf0550 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34a04db02b7148da89f56b89dc1921eb, user_id=56b6a40596ce4c0fab0cf7def22fbc54, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:37.920193 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952dd0f2-12f9-425c-ba85-efb831bf0550 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:37.933941 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 815/1606] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:15:37 2022] PUT /identity/v3/projects/1443b802dc724950b5eb5c6446e2515c/users/4206a21c317e4b5aace8541699f8032d/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:15:37.933941 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a0ce8-0344-4e45-a678-d58729ac1a56 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:37.961434 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 792/1607] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:15:37 2022] PUT /identity/v3/projects/34a04db02b7148da89f56b89dc1921eb/users/56b6a40596ce4c0fab0cf7def22fbc54/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:15:37.963652 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1b8fc-10a1-41fc-b43d-3453a8631083 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:37.963992 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1b8fc-10a1-41fc-b43d-3453a8631083 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:37.964323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1b8fc-10a1-41fc-b43d-3453a8631083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:38.001088 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:38.007006 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-10f1b8fc-10a1-41fc-b43d-3453a8631083 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:38.008735 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-10f1b8fc-10a1-41fc-b43d-3453a8631083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:38.029321 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a0ce8-0344-4e45-a678-d58729ac1a56 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:38.089809 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 793/1608] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:38.094198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b5cbe6-4c14-4360-8c2c-bb65e75fd054 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:38.094504 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b5cbe6-4c14-4360-8c2c-bb65e75fd054 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:38.095439 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b5cbe6-4c14-4360-8c2c-bb65e75fd054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:38.109968 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b5800-0f0a-4188-89f7-3dad1f8387e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:38.111082 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b5800-0f0a-4188-89f7-3dad1f8387e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:38.111474 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b5800-0f0a-4188-89f7-3dad1f8387e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:38.111755 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b5800-0f0a-4188-89f7-3dad1f8387e3 admin admin] PATH_INFO: `/v3/projects/73d7613037564f069b2c63dbde4144d8/users/c100a64038f44a7ea2324701e9526943/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:38.128224 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b5800-0f0a-4188-89f7-3dad1f8387e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73d7613037564f069b2c63dbde4144d8, user_id=c100a64038f44a7ea2324701e9526943, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:38.131167 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b5800-0f0a-4188-89f7-3dad1f8387e3 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:38.135613 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b8b5cbe6-4c14-4360-8c2c-bb65e75fd054 None None] MFA Rules not processed for user `866c56c184f14dde80e90670dc3e758f`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:38.137090 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b8b5cbe6-4c14-4360-8c2c-bb65e75fd054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:38.151720 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 816/1609] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:15:37 2022] PUT /identity/v3/projects/73d7613037564f069b2c63dbde4144d8/users/c100a64038f44a7ea2324701e9526943/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:15:38.159527 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b5800-0f0a-4188-89f7-3dad1f8387e3 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:38.162189 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:38.279944 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b5800-0f0a-4188-89f7-3dad1f8387e3 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:38.281173 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 794/1610] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:38 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:38.285462 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c654d0-bbcb-4582-b208-072945174605 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:38.285948 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c654d0-bbcb-4582-b208-072945174605 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:38.286401 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c654d0-bbcb-4582-b208-072945174605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:38.287147 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7850c-5d42-46ce-a2f7-21ae0f7fe3e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:38.287996 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7850c-5d42-46ce-a2f7-21ae0f7fe3e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:38.288347 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7850c-5d42-46ce-a2f7-21ae0f7fe3e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:38.288699 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7850c-5d42-46ce-a2f7-21ae0f7fe3e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:38.316641 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-79c654d0-bbcb-4582-b208-072945174605 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:38.317944 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-79c654d0-bbcb-4582-b208-072945174605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:38.386894 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7850c-5d42-46ce-a2f7-21ae0f7fe3e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:38.388923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7850c-5d42-46ce-a2f7-21ae0f7fe3e2 service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:38.408401 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 795/1611] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:38.408401 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e5c2b-6f02-486b-a4da-a88dd7a92b4a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:38.408401 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e5c2b-6f02-486b-a4da-a88dd7a92b4a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:38.408401 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e5c2b-6f02-486b-a4da-a88dd7a92b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:38.412560 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 817/1612] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:15:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:15:38.413268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7850c-5d42-46ce-a2f7-21ae0f7fe3e2 service neutron] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:38.426268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7850c-5d42-46ce-a2f7-21ae0f7fe3e2 service neutron] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:38.429974 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66b3e08-8dac-4920-b29f-bdf0b04f0df7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:38.430627 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b3e08-8dac-4920-b29f-bdf0b04f0df7 service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:38.430907 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b3e08-8dac-4920-b29f-bdf0b04f0df7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:38.431215 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b3e08-8dac-4920-b29f-bdf0b04f0df7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:38.441288 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66b3e08-8dac-4920-b29f-bdf0b04f0df7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:38.442708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66b3e08-8dac-4920-b29f-bdf0b04f0df7 service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:38.451130 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-7a1e5c2b-6f02-486b-a4da-a88dd7a92b4a None None] MFA Rules not processed for user `56b6a40596ce4c0fab0cf7def22fbc54`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:38.452662 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7a1e5c2b-6f02-486b-a4da-a88dd7a92b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:38.453603 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 818/1613] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:15:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:15:38.454661 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7baba6-0bdf-40fe-82df-fa506513dbbf None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:38.454944 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7baba6-0bdf-40fe-82df-fa506513dbbf None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:38.455214 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7baba6-0bdf-40fe-82df-fa506513dbbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:38.492837 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-1d7baba6-0bdf-40fe-82df-fa506513dbbf None None] MFA Rules not processed for user `c100a64038f44a7ea2324701e9526943`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:38.494637 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-1d7baba6-0bdf-40fe-82df-fa506513dbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:38.502900 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:38.558585 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 796/1614] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:38 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:38.560715 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e5c2b-6f02-486b-a4da-a88dd7a92b4a None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:38.581848 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e5c2b-6f02-486b-a4da-a88dd7a92b4a None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:38.597430 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507576b-1462-4d8a-8a97-ad2eb643543c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:38.598266 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507576b-1462-4d8a-8a97-ad2eb643543c admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:38.598655 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507576b-1462-4d8a-8a97-ad2eb643543c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:38.599031 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507576b-1462-4d8a-8a97-ad2eb643543c admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:38.601125 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c507576b-1462-4d8a-8a97-ad2eb643543c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:38.607157 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c507576b-1462-4d8a-8a97-ad2eb643543c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:38.614644 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 819/1615] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:38 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:38.615539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7baba6-0bdf-40fe-82df-fa506513dbbf None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:38.628564 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7baba6-0bdf-40fe-82df-fa506513dbbf None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:38.633196 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17445f6-6ddc-4b0f-a397-aa61fd354636 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:38.634053 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17445f6-6ddc-4b0f-a397-aa61fd354636 service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:38.634423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17445f6-6ddc-4b0f-a397-aa61fd354636 service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:38.634855 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17445f6-6ddc-4b0f-a397-aa61fd354636 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:38.648066 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17445f6-6ddc-4b0f-a397-aa61fd354636 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:38.649882 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17445f6-6ddc-4b0f-a397-aa61fd354636 service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:38.655739 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 797/1616] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:15:38 2022] POST /identity/v3/projects => generated 365 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:15:38.656573 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507576b-1462-4d8a-8a97-ad2eb643543c admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:38.668589 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 820/1617] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:15:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:15:38.669727 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17445f6-6ddc-4b0f-a397-aa61fd354636 service neutron] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:38.681190 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17445f6-6ddc-4b0f-a397-aa61fd354636 service neutron] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:38.703500 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:38.839624 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85a6cb9-5a18-4ba4-8f4d-f8d90dd00892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:38.839935 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507576b-1462-4d8a-8a97-ad2eb643543c admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:38.841515 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a6cb9-5a18-4ba4-8f4d-f8d90dd00892 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:38.842061 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a6cb9-5a18-4ba4-8f4d-f8d90dd00892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:38.842561 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a6cb9-5a18-4ba4-8f4d-f8d90dd00892 admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:38.844543 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85a6cb9-5a18-4ba4-8f4d-f8d90dd00892 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:38.859415 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85a6cb9-5a18-4ba4-8f4d-f8d90dd00892 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:38.863674 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac4db1-ae26-409d-9fe9-1f793a81dfe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:38.867799 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac4db1-ae26-409d-9fe9-1f793a81dfe7 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:38.868165 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac4db1-ae26-409d-9fe9-1f793a81dfe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:38.868459 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac4db1-ae26-409d-9fe9-1f793a81dfe7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:38.874311 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac4db1-ae26-409d-9fe9-1f793a81dfe7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:38.878665 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac4db1-ae26-409d-9fe9-1f793a81dfe7 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:38.915423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 821/1618] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:15:38 2022] POST /identity/v3/projects => generated 405 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:15:38.920062 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85a6cb9-5a18-4ba4-8f4d-f8d90dd00892 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:38.952662 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 798/1619] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:15:38 2022] POST /identity/v3/projects => generated 367 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:15:38.956938 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac4db1-ae26-409d-9fe9-1f793a81dfe7 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:39.062335 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85a6cb9-5a18-4ba4-8f4d-f8d90dd00892 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:39.122352 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:39.176156 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac4db1-ae26-409d-9fe9-1f793a81dfe7 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:39.186515 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5136486-598f-4123-9403-afdd002bb609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:39.188160 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f701ca6-936a-4b46-8fdd-7b099cac4455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:39.188430 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5136486-598f-4123-9403-afdd002bb609 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:39.188719 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5136486-598f-4123-9403-afdd002bb609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:39.189583 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5136486-598f-4123-9403-afdd002bb609 admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:39.190033 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f701ca6-936a-4b46-8fdd-7b099cac4455 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:39.190314 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f701ca6-936a-4b46-8fdd-7b099cac4455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:39.190587 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f701ca6-936a-4b46-8fdd-7b099cac4455 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:39.191521 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5136486-598f-4123-9403-afdd002bb609 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:39.193033 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f701ca6-936a-4b46-8fdd-7b099cac4455 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:39.196405 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5136486-598f-4123-9403-afdd002bb609 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:39.197834 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f701ca6-936a-4b46-8fdd-7b099cac4455 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:39.268248 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 822/1620] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:15:38 2022] POST /identity/v3/users => generated 289 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:15:39.271505 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5136486-598f-4123-9403-afdd002bb609 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:39.273052 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 799/1621] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:15:38 2022] POST /identity/v3/users => generated 309 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:15:39.273904 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f701ca6-936a-4b46-8fdd-7b099cac4455 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:39.297898 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5136486-598f-4123-9403-afdd002bb609 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:39.301665 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f701ca6-936a-4b46-8fdd-7b099cac4455 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:39.305948 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3bebdd-d7e9-4a5b-963c-9bb425fb1109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:39.306643 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bebdd-d7e9-4a5b-963c-9bb425fb1109 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:39.306928 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bebdd-d7e9-4a5b-963c-9bb425fb1109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:39.307528 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bebdd-d7e9-4a5b-963c-9bb425fb1109 admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:39.308204 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3bebdd-d7e9-4a5b-963c-9bb425fb1109 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:39.310205 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3bebdd-d7e9-4a5b-963c-9bb425fb1109 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:39.315947 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07176925-62f9-4117-911f-bb2d4ddb5119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:39.316635 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07176925-62f9-4117-911f-bb2d4ddb5119 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:39.316922 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07176925-62f9-4117-911f-bb2d4ddb5119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:39.317226 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07176925-62f9-4117-911f-bb2d4ddb5119 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:39.317958 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07176925-62f9-4117-911f-bb2d4ddb5119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:39.319616 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07176925-62f9-4117-911f-bb2d4ddb5119 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:39.340200 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 823/1622] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:15:39 2022] POST /identity/v3/users => generated 290 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:15:39.345248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 800/1623] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:15:39.346707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3bebdd-d7e9-4a5b-963c-9bb425fb1109 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:39.347555 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07176925-62f9-4117-911f-bb2d4ddb5119 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:39.368835 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07176925-62f9-4117-911f-bb2d4ddb5119 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:39.374308 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3bebdd-d7e9-4a5b-963c-9bb425fb1109 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:39.374553 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ed018-c660-4bf4-9b3c-9f00cf7bc49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:39.375285 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ed018-c660-4bf4-9b3c-9f00cf7bc49d admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:39.375632 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ed018-c660-4bf4-9b3c-9f00cf7bc49d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:39.375913 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ed018-c660-4bf4-9b3c-9f00cf7bc49d admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:39.376644 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ed018-c660-4bf4-9b3c-9f00cf7bc49d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:39.378259 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ed018-c660-4bf4-9b3c-9f00cf7bc49d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:39.381027 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec75e9-f0d1-4a5c-b55f-d4a08621f9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:39.381978 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec75e9-f0d1-4a5c-b55f-d4a08621f9e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:39.382420 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec75e9-f0d1-4a5c-b55f-d4a08621f9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:39.383131 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec75e9-f0d1-4a5c-b55f-d4a08621f9e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:39.384514 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fec75e9-f0d1-4a5c-b55f-d4a08621f9e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:39.386450 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fec75e9-f0d1-4a5c-b55f-d4a08621f9e0 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:39.400595 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 801/1624] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:15:39.401475 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ed018-c660-4bf4-9b3c-9f00cf7bc49d admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:39.403851 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 824/1625] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:15:39.406632 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec75e9-f0d1-4a5c-b55f-d4a08621f9e0 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:39.414003 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ed018-c660-4bf4-9b3c-9f00cf7bc49d admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:39.421333 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eaa5a6-2c19-4ff1-9af7-a059f2b98d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:39.422239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eaa5a6-2c19-4ff1-9af7-a059f2b98d10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:39.422239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eaa5a6-2c19-4ff1-9af7-a059f2b98d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:39.422474 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eaa5a6-2c19-4ff1-9af7-a059f2b98d10 admin admin] PATH_INFO: `/v3/projects/18b55e0245c143dba1be592294e9f8af/users/bbc35976c35f42eba4e29f69c3f9f8b3/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:39.423359 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec75e9-f0d1-4a5c-b55f-d4a08621f9e0 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:39.432252 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb4118-252b-4b1b-a270-b43c9d202605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:39.433809 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb4118-252b-4b1b-a270-b43c9d202605 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:39.434672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb4118-252b-4b1b-a270-b43c9d202605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:39.435279 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb4118-252b-4b1b-a270-b43c9d202605 admin admin] PATH_INFO: `/v3/projects/fe9066ff903c4d91b49e294a4b9c9817/users/829fa2e7304c4a33920a7194a81cc9e6/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:39.444573 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eaa5a6-2c19-4ff1-9af7-a059f2b98d10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18b55e0245c143dba1be592294e9f8af, user_id=bbc35976c35f42eba4e29f69c3f9f8b3, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:39.447759 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eaa5a6-2c19-4ff1-9af7-a059f2b98d10 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:39.464100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cb4118-252b-4b1b-a270-b43c9d202605 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe9066ff903c4d91b49e294a4b9c9817, user_id=829fa2e7304c4a33920a7194a81cc9e6, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:39.468660 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cb4118-252b-4b1b-a270-b43c9d202605 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:39.471506 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 802/1626] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:15:39 2022] PUT /identity/v3/projects/18b55e0245c143dba1be592294e9f8af/users/bbc35976c35f42eba4e29f69c3f9f8b3/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:15:39.473594 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eaa5a6-2c19-4ff1-9af7-a059f2b98d10 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:39.508167 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 825/1627] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:15:39 2022] PUT /identity/v3/projects/fe9066ff903c4d91b49e294a4b9c9817/users/829fa2e7304c4a33920a7194a81cc9e6/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:15:39.510621 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb4118-252b-4b1b-a270-b43c9d202605 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:39.547378 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:39.561114 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eaa5a6-2c19-4ff1-9af7-a059f2b98d10 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:39.687687 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb4118-252b-4b1b-a270-b43c9d202605 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:39.698782 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eaadba-f713-43f1-bca4-76a12ea6288e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:39.699635 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a16b2-6524-4b69-9af2-1967e153fdef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:39.701020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a16b2-6524-4b69-9af2-1967e153fdef admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:39.701671 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eaadba-f713-43f1-bca4-76a12ea6288e admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:39.702023 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eaadba-f713-43f1-bca4-76a12ea6288e admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:39.702367 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eaadba-f713-43f1-bca4-76a12ea6288e admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:39.702711 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a16b2-6524-4b69-9af2-1967e153fdef admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:39.705145 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a16b2-6524-4b69-9af2-1967e153fdef admin admin] PATH_INFO: `/v3/projects/d648cc83af4a453b8f1472d9a1c58464/users/0f8e0b8f9b1d48a8b85b969ed13bffd6/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:39.706146 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eaadba-f713-43f1-bca4-76a12ea6288e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:39.707885 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eaadba-f713-43f1-bca4-76a12ea6288e admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:39.738755 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 826/1628] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:15:39.747688 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eaadba-f713-43f1-bca4-76a12ea6288e admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:39.749589 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a16b2-6524-4b69-9af2-1967e153fdef admin admin] RBAC: Authorizing `identity:create_grant(project_id=d648cc83af4a453b8f1472d9a1c58464, user_id=0f8e0b8f9b1d48a8b85b969ed13bffd6, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:39.757972 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a16b2-6524-4b69-9af2-1967e153fdef admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:39.769289 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eaadba-f713-43f1-bca4-76a12ea6288e admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:39.787371 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 803/1629] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:15:39 2022] PUT /identity/v3/projects/d648cc83af4a453b8f1472d9a1c58464/users/0f8e0b8f9b1d48a8b85b969ed13bffd6/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:15:39.787371 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a16b2-6524-4b69-9af2-1967e153fdef admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:39.906945 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d5c4f-51f2-46f4-acf0-09718c3be719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:39.908606 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d5c4f-51f2-46f4-acf0-09718c3be719 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:39.909380 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d5c4f-51f2-46f4-acf0-09718c3be719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:39.910092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d5c4f-51f2-46f4-acf0-09718c3be719 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:39.911130 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1d5c4f-51f2-46f4-acf0-09718c3be719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:39.913477 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1d5c4f-51f2-46f4-acf0-09718c3be719 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:39.915649 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:39.928614 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a16b2-6524-4b69-9af2-1967e153fdef admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:39.930712 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 827/1630] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:15:39.932655 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d5c4f-51f2-46f4-acf0-09718c3be719 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:39.948119 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29fa61-767c-454e-bd77-15f1cb6e4543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:39.949010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d5c4f-51f2-46f4-acf0-09718c3be719 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:39.949151 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29fa61-767c-454e-bd77-15f1cb6e4543 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:39.949401 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29fa61-767c-454e-bd77-15f1cb6e4543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:39.949636 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29fa61-767c-454e-bd77-15f1cb6e4543 admin admin] PATH_INFO: `/v3/projects/18b55e0245c143dba1be592294e9f8af/users/bbc35976c35f42eba4e29f69c3f9f8b3/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:39.954961 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e29fa61-767c-454e-bd77-15f1cb6e4543 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18b55e0245c143dba1be592294e9f8af, user_id=bbc35976c35f42eba4e29f69c3f9f8b3, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:39.957665 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e29fa61-767c-454e-bd77-15f1cb6e4543 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:39.958151 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab1f1e4-f998-47f3-9d6e-6fc554443f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:39.958995 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1f1e4-f998-47f3-9d6e-6fc554443f79 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:39.959466 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1f1e4-f998-47f3-9d6e-6fc554443f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:39.959844 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1f1e4-f998-47f3-9d6e-6fc554443f79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:39.961346 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab1f1e4-f998-47f3-9d6e-6fc554443f79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:39.963109 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab1f1e4-f998-47f3-9d6e-6fc554443f79 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:39.980444 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 828/1631] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:15:39.981998 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab1f1e4-f998-47f3-9d6e-6fc554443f79 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:39.984864 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 804/1632] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:15:39 2022] PUT /identity/v3/projects/18b55e0245c143dba1be592294e9f8af/users/bbc35976c35f42eba4e29f69c3f9f8b3/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:15:39.986782 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29fa61-767c-454e-bd77-15f1cb6e4543 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:39.996202 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab1f1e4-f998-47f3-9d6e-6fc554443f79 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:40.109857 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29fa61-767c-454e-bd77-15f1cb6e4543 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:40.112627 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe8bbe8-96f4-4bac-aff6-060174720b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:40.113806 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8bbe8-96f4-4bac-aff6-060174720b53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:40.114596 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8bbe8-96f4-4bac-aff6-060174720b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:40.115079 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8bbe8-96f4-4bac-aff6-060174720b53 admin admin] PATH_INFO: `/v3/projects/fe9066ff903c4d91b49e294a4b9c9817/users/829fa2e7304c4a33920a7194a81cc9e6/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:40.115666 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376be8b-2b33-4441-b4dd-f99ac4bba7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:40.116845 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376be8b-2b33-4441-b4dd-f99ac4bba7cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:40.117226 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376be8b-2b33-4441-b4dd-f99ac4bba7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:40.117636 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376be8b-2b33-4441-b4dd-f99ac4bba7cf admin admin] PATH_INFO: `/v3/projects/d648cc83af4a453b8f1472d9a1c58464/users/0f8e0b8f9b1d48a8b85b969ed13bffd6/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:40.120115 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe8bbe8-96f4-4bac-aff6-060174720b53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe9066ff903c4d91b49e294a4b9c9817, user_id=829fa2e7304c4a33920a7194a81cc9e6, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:40.120276 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e376be8b-2b33-4441-b4dd-f99ac4bba7cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d648cc83af4a453b8f1472d9a1c58464, user_id=0f8e0b8f9b1d48a8b85b969ed13bffd6, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:40.123163 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe8bbe8-96f4-4bac-aff6-060174720b53 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:40.123774 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e376be8b-2b33-4441-b4dd-f99ac4bba7cf admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:40.144325 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 805/1633] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:15:39 2022] PUT /identity/v3/projects/d648cc83af4a453b8f1472d9a1c58464/users/0f8e0b8f9b1d48a8b85b969ed13bffd6/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:15:40.144325 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e1028-2dea-4ec2-aff4-fc71cdb0fced None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:40.144325 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e1028-2dea-4ec2-aff4-fc71cdb0fced None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:40.144325 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e1028-2dea-4ec2-aff4-fc71cdb0fced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:40.155903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 829/1634] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:15:39 2022] PUT /identity/v3/projects/fe9066ff903c4d91b49e294a4b9c9817/users/829fa2e7304c4a33920a7194a81cc9e6/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:15:40.155903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b122d7-baaf-4376-aeb0-1c97495ca104 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:40.155903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b122d7-baaf-4376-aeb0-1c97495ca104 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:40.155903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b122d7-baaf-4376-aeb0-1c97495ca104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:40.173159 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:40.182962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-2f1e1028-2dea-4ec2-aff4-fc71cdb0fced None None] MFA Rules not processed for user `bbc35976c35f42eba4e29f69c3f9f8b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:40.187222 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2f1e1028-2dea-4ec2-aff4-fc71cdb0fced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:40.233036 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-16b122d7-baaf-4376-aeb0-1c97495ca104 None None] MFA Rules not processed for user `0f8e0b8f9b1d48a8b85b969ed13bffd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:40.236368 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-16b122d7-baaf-4376-aeb0-1c97495ca104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:40.311418 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 806/1635] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:40 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:40.313057 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1295a8-2055-48aa-af31-14924f4854c2 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:40.319710 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1295a8-2055-48aa-af31-14924f4854c2 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:40.320695 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1295a8-2055-48aa-af31-14924f4854c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:40.384036 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 830/1636] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:40 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:40.384036 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b122d7-baaf-4376-aeb0-1c97495ca104 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:40.385120 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-fb1295a8-2055-48aa-af31-14924f4854c2 None None] MFA Rules not processed for user `829fa2e7304c4a33920a7194a81cc9e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:40.386742 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-fb1295a8-2055-48aa-af31-14924f4854c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:40.491915 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b122d7-baaf-4376-aeb0-1c97495ca104 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:40.495365 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97569cb-d114-47ea-bd28-188bba815757 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:40.496328 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97569cb-d114-47ea-bd28-188bba815757 service cinder] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:40.496458 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97569cb-d114-47ea-bd28-188bba815757 service cinder] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:40.497573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97569cb-d114-47ea-bd28-188bba815757 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:40.507874 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97569cb-d114-47ea-bd28-188bba815757 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:40.509256 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97569cb-d114-47ea-bd28-188bba815757 service cinder] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:40.522621 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 831/1637] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:15:40 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:15:40.523695 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97569cb-d114-47ea-bd28-188bba815757 service cinder] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:40.535944 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97569cb-d114-47ea-bd28-188bba815757 service cinder] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:40.540328 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeac0c0-a85d-4646-8738-cac6d08aef07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:40.542193 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeac0c0-a85d-4646-8738-cac6d08aef07 service cinder] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:40.542626 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeac0c0-a85d-4646-8738-cac6d08aef07 service cinder] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:40.543160 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeac0c0-a85d-4646-8738-cac6d08aef07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:40.550426 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 807/1638] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:40 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:40.551203 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:40.562710 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeac0c0-a85d-4646-8738-cac6d08aef07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:40.565240 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeac0c0-a85d-4646-8738-cac6d08aef07 service cinder] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:40.578737 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 832/1639] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:15:40 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:15:40.593760 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1295a8-2055-48aa-af31-14924f4854c2 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:40.607241 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1295a8-2055-48aa-af31-14924f4854c2 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:40.619578 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66171291-5b28-4544-97bf-c754f08d34ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:40.620918 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66171291-5b28-4544-97bf-c754f08d34ab service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:40.621356 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66171291-5b28-4544-97bf-c754f08d34ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:40.621678 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66171291-5b28-4544-97bf-c754f08d34ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:40.637178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66171291-5b28-4544-97bf-c754f08d34ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:40.639809 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66171291-5b28-4544-97bf-c754f08d34ab service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:40.671753 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 808/1640] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:15:40 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:15:40.754790 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b45b0-0ae3-4abd-a1da-93c8511141b2 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:40.756056 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b45b0-0ae3-4abd-a1da-93c8511141b2 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:40.756868 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b45b0-0ae3-4abd-a1da-93c8511141b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:40.820917 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-765b45b0-0ae3-4abd-a1da-93c8511141b2 None None] MFA Rules not processed for user `34ca0974e6dd49bdb6a56ec577dad3cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:40.833628 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66171291-5b28-4544-97bf-c754f08d34ab service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:40.841268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-765b45b0-0ae3-4abd-a1da-93c8511141b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:40.843702 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66171291-5b28-4544-97bf-c754f08d34ab service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:40.854273 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8607f6c-91af-4109-b572-6b1af3e4985d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:40.855258 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8607f6c-91af-4109-b572-6b1af3e4985d service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:40.855785 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8607f6c-91af-4109-b572-6b1af3e4985d service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:40.857173 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8607f6c-91af-4109-b572-6b1af3e4985d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:40.999885 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8607f6c-91af-4109-b572-6b1af3e4985d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:41.002147 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8607f6c-91af-4109-b572-6b1af3e4985d service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:41.007425 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 833/1641] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:41.008528 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b45b0-0ae3-4abd-a1da-93c8511141b2 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:41.026891 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 809/1642] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:15:40 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 16:15:41.029821 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b45b0-0ae3-4abd-a1da-93c8511141b2 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:41.032064 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc86b5-6c62-44b5-8d37-c9adf1932f6a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:41.032409 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc86b5-6c62-44b5-8d37-c9adf1932f6a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:41.032700 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc86b5-6c62-44b5-8d37-c9adf1932f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:41.038024 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81270c62-eb47-49f2-8fc4-db883f37da02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:41.038698 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81270c62-eb47-49f2-8fc4-db883f37da02 service cinder] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:41.039179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81270c62-eb47-49f2-8fc4-db883f37da02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:41.039510 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81270c62-eb47-49f2-8fc4-db883f37da02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:41.069147 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:41.090841 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-e0fc86b5-6c62-44b5-8d37-c9adf1932f6a None None] MFA Rules not processed for user `34ca0974e6dd49bdb6a56ec577dad3cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:41.094489 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e0fc86b5-6c62-44b5-8d37-c9adf1932f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:41.141120 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 810/1643] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:41.176714 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81270c62-eb47-49f2-8fc4-db883f37da02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:41.178191 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81270c62-eb47-49f2-8fc4-db883f37da02 service cinder] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:41.197123 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f42b0-ed66-4f95-a56e-f4be8936a52e None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:41.197590 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f42b0-ed66-4f95-a56e-f4be8936a52e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:41.198654 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f42b0-ed66-4f95-a56e-f4be8936a52e None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:41.201487 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 811/1644] 10.222.0.41 () {54 vars in 961 bytes} [Fri Jun 3 16:15:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 16:15:41.204247 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 834/1645] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:15:41 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:15:41.255515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480d6ca-4a2c-48f6-8164-4a6c52e1e266 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:41.255688 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480d6ca-4a2c-48f6-8164-4a6c52e1e266 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:41.256690 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480d6ca-4a2c-48f6-8164-4a6c52e1e266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:41.295223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-7480d6ca-4a2c-48f6-8164-4a6c52e1e266 None None] MFA Rules not processed for user `2fd08ea39ee94b92ac6763608e69f158`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:41.296604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7480d6ca-4a2c-48f6-8164-4a6c52e1e266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:41.491115 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 812/1646] 10.222.0.41 () {58 vars in 1058 bytes} [Fri Jun 3 16:15:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:41.491960 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81270c62-eb47-49f2-8fc4-db883f37da02 service cinder] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:41.511014 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81270c62-eb47-49f2-8fc4-db883f37da02 service cinder] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:41.522705 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6f02f-ce87-44ec-92fb-3bda77f69da1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:41.523363 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6f02f-ce87-44ec-92fb-3bda77f69da1 service nova] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:41.523657 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6f02f-ce87-44ec-92fb-3bda77f69da1 service nova] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:41.523939 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6f02f-ce87-44ec-92fb-3bda77f69da1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:41.538700 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a6f02f-ce87-44ec-92fb-3bda77f69da1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:41.540497 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a6f02f-ce87-44ec-92fb-3bda77f69da1 service nova] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:41.553551 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 835/1647] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:15:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 16:15:42.040302 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7480d6ca-4a2c-48f6-8164-4a6c52e1e266 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:42.183377 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7480d6ca-4a2c-48f6-8164-4a6c52e1e266 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:42.194812 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d52bd-b849-4bc8-86d5-a7e80039fe58 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:42.195103 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d52bd-b849-4bc8-86d5-a7e80039fe58 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:42.195424 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d52bd-b849-4bc8-86d5-a7e80039fe58 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:42.196859 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 836/1648] 10.222.0.41 () {54 vars in 912 bytes} [Fri Jun 3 16:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 16:15:42.203942 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553de1c-c4b7-4822-8839-616d3c9f71da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:42.204854 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553de1c-c4b7-4822-8839-616d3c9f71da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:42.205142 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553de1c-c4b7-4822-8839-616d3c9f71da admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:42.205424 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553de1c-c4b7-4822-8839-616d3c9f71da admin admin] PATH_INFO: `/v3/users/c100a64038f44a7ea2324701e9526943` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:42.208326 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca3f327-c4e1-41e2-a705-48743050e39b None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:42.208618 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca3f327-c4e1-41e2-a705-48743050e39b None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:42.208899 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca3f327-c4e1-41e2-a705-48743050e39b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:42.210097 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3553de1c-c4b7-4822-8839-616d3c9f71da admin admin] RBAC: Authorizing `identity:delete_user(user_id=c100a64038f44a7ea2324701e9526943)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:42.212368 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3553de1c-c4b7-4822-8839-616d3c9f71da admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:42.245540 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-eca3f327-c4e1-41e2-a705-48743050e39b None None] MFA Rules not processed for user `2fd08ea39ee94b92ac6763608e69f158`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:42.247373 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-eca3f327-c4e1-41e2-a705-48743050e39b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:42.273220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 837/1649] 10.222.0.41 () {58 vars in 1009 bytes} [Fri Jun 3 16:15:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:15:42.293379 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca3f327-c4e1-41e2-a705-48743050e39b None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:42.295453 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-eca3f327-c4e1-41e2-a705-48743050e39b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:42.313577 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3553de1c-c4b7-4822-8839-616d3c9f71da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c100a64038f44a7ea2324701e9526943'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:42.329118 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3553de1c-c4b7-4822-8839-616d3c9f71da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c100a64038f44a7ea2324701e9526943'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:42.352502 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3553de1c-c4b7-4822-8839-616d3c9f71da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c100a64038f44a7ea2324701e9526943'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:42.353472 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3553de1c-c4b7-4822-8839-616d3c9f71da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c100a64038f44a7ea2324701e9526943'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:42.368471 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3553de1c-c4b7-4822-8839-616d3c9f71da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c100a64038f44a7ea2324701e9526943'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:42.380931 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 813/1650] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:15:42 2022] DELETE /identity/v3/users/c100a64038f44a7ea2324701e9526943 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:15:42.385731 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584add3-03b3-46c4-b881-a0177ff71e15 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:42.386219 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584add3-03b3-46c4-b881-a0177ff71e15 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:42.386824 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584add3-03b3-46c4-b881-a0177ff71e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:42.438189 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-3584add3-03b3-46c4-b881-a0177ff71e15 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:42.440727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3584add3-03b3-46c4-b881-a0177ff71e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:42.462393 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:42.525183 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca3f327-c4e1-41e2-a705-48743050e39b None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:42.576057 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:42.614928 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 814/1651] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:42.661312 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3584add3-03b3-46c4-b881-a0177ff71e15 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:42.663978 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3584add3-03b3-46c4-b881-a0177ff71e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:42.719148 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d00251-6d5e-46f3-bcfe-6fec9894cbfc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbaf66d6e1f54047917cc615546bbd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:42.721063 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d00251-6d5e-46f3-bcfe-6fec9894cbfc service glance] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:42.721363 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d00251-6d5e-46f3-bcfe-6fec9894cbfc service glance] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:42.721655 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d00251-6d5e-46f3-bcfe-6fec9894cbfc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:42.725217 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-b1d00251-6d5e-46f3-bcfe-6fec9894cbfc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:42.798604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3584add3-03b3-46c4-b881-a0177ff71e15 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:42.804143 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608785b5-4afd-48ad-8803-58ae377ca08b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:42.805034 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608785b5-4afd-48ad-8803-58ae377ca08b service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:42.805428 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608785b5-4afd-48ad-8803-58ae377ca08b service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:42.806634 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608785b5-4afd-48ad-8803-58ae377ca08b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:42.831107 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608785b5-4afd-48ad-8803-58ae377ca08b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:42.832785 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608785b5-4afd-48ad-8803-58ae377ca08b service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:42.852364 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 815/1652] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:15:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 16:15:42.881578 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d00251-6d5e-46f3-bcfe-6fec9894cbfc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:42.883236 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d00251-6d5e-46f3-bcfe-6fec9894cbfc service glance] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:42.913173 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 838/1653] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:15:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 16:15:44.307074 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2411cb1-500f-4795-8766-2e57c94d3263 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:44.307682 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2411cb1-500f-4795-8766-2e57c94d3263 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:44.308072 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2411cb1-500f-4795-8766-2e57c94d3263 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:44.310764 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 816/1654] 10.222.0.41 () {54 vars in 912 bytes} [Fri Jun 3 16:15:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:15:44.317260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7ff67-5d35-4cda-8b27-a22e36c7215b None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:44.317739 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7ff67-5d35-4cda-8b27-a22e36c7215b None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:44.321957 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7ff67-5d35-4cda-8b27-a22e36c7215b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:44.418200 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-e6d7ff67-5d35-4cda-8b27-a22e36c7215b None None] MFA Rules not processed for user `aeeaad0680074179b3d29abcf0d51515`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:44.420140 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-e6d7ff67-5d35-4cda-8b27-a22e36c7215b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:44.438209 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 839/1655] 10.222.0.41 () {58 vars in 1009 bytes} [Fri Jun 3 16:15:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:44.656427 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2411cb1-500f-4795-8766-2e57c94d3263 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:44.658591 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d2411cb1-500f-4795-8766-2e57c94d3263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:44.689740 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2411cb1-500f-4795-8766-2e57c94d3263 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:44.706830 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30e21a-1dcf-47b2-8b60-393c14ba79fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:44.707718 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30e21a-1dcf-47b2-8b60-393c14ba79fb service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:44.708123 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30e21a-1dcf-47b2-8b60-393c14ba79fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:44.708513 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30e21a-1dcf-47b2-8b60-393c14ba79fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:44.726053 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb30e21a-1dcf-47b2-8b60-393c14ba79fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:44.731480 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb30e21a-1dcf-47b2-8b60-393c14ba79fb service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:44.753106 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 817/1656] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:15:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 16:15:45.559120 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7ff67-5d35-4cda-8b27-a22e36c7215b None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:45.563144 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-e6d7ff67-5d35-4cda-8b27-a22e36c7215b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:45.624184 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7ff67-5d35-4cda-8b27-a22e36c7215b None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:45.668826 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258958ce-e505-4389-a76d-75f262b5978f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:45.672422 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258958ce-e505-4389-a76d-75f262b5978f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:45.676311 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258958ce-e505-4389-a76d-75f262b5978f admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:45.677928 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258958ce-e505-4389-a76d-75f262b5978f admin admin] PATH_INFO: `/v3/projects/73d7613037564f069b2c63dbde4144d8` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:45.681278 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258958ce-e505-4389-a76d-75f262b5978f admin admin] RBAC: Authorizing `identity:delete_project(project_id=73d7613037564f069b2c63dbde4144d8)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:45.685355 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258958ce-e505-4389-a76d-75f262b5978f admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:45.885830 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-258958ce-e505-4389-a76d-75f262b5978f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73d7613037564f069b2c63dbde4144d8', 'request_id': 'req-258958ce-e505-4389-a76d-75f262b5978f'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:45.917874 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-258958ce-e505-4389-a76d-75f262b5978f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73d7613037564f069b2c63dbde4144d8', 'request_id': 'req-258958ce-e505-4389-a76d-75f262b5978f'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:45.935175 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-258958ce-e505-4389-a76d-75f262b5978f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73d7613037564f069b2c63dbde4144d8', 'request_id': 'req-258958ce-e505-4389-a76d-75f262b5978f'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:45.957779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-258958ce-e505-4389-a76d-75f262b5978f admin admin] The token cache is being invalidate because project 73d7613037564f069b2c63dbde4144d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:15:45.959328 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-258958ce-e505-4389-a76d-75f262b5978f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:45.960430 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-258958ce-e505-4389-a76d-75f262b5978f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:45.963412 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 840/1657] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:15:45 2022] DELETE /identity/v3/projects/73d7613037564f069b2c63dbde4144d8 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:15:45.968632 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30e21a-1dcf-47b2-8b60-393c14ba79fb service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:45.999254 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30e21a-1dcf-47b2-8b60-393c14ba79fb service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:46.128062 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79293897-886f-4c72-b9c8-c5f3f218b078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:46.129968 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79293897-886f-4c72-b9c8-c5f3f218b078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:46.130458 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79293897-886f-4c72-b9c8-c5f3f218b078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:46.130845 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79293897-886f-4c72-b9c8-c5f3f218b078 admin admin] PATH_INFO: `/v3/users/0f8e0b8f9b1d48a8b85b969ed13bffd6` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:46.136111 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79293897-886f-4c72-b9c8-c5f3f218b078 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f8e0b8f9b1d48a8b85b969ed13bffd6)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:46.140511 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79293897-886f-4c72-b9c8-c5f3f218b078 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:46.232796 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:46.244152 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-79293897-886f-4c72-b9c8-c5f3f218b078 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f8e0b8f9b1d48a8b85b969ed13bffd6'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:46.258253 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-79293897-886f-4c72-b9c8-c5f3f218b078 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f8e0b8f9b1d48a8b85b969ed13bffd6'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:46.318527 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-79293897-886f-4c72-b9c8-c5f3f218b078 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f8e0b8f9b1d48a8b85b969ed13bffd6'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:46.322281 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-79293897-886f-4c72-b9c8-c5f3f218b078 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f8e0b8f9b1d48a8b85b969ed13bffd6'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:46.338461 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258958ce-e505-4389-a76d-75f262b5978f admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:46.340384 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-258958ce-e505-4389-a76d-75f262b5978f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:46.367858 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-79293897-886f-4c72-b9c8-c5f3f218b078 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f8e0b8f9b1d48a8b85b969ed13bffd6'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:46.372549 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258958ce-e505-4389-a76d-75f262b5978f admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:46.399205 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 818/1658] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:15:45 2022] DELETE /identity/v3/users/0f8e0b8f9b1d48a8b85b969ed13bffd6 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:15:46.401245 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428aa488-19d8-4de6-aff4-ccaf77d5a162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:46.401512 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79293897-886f-4c72-b9c8-c5f3f218b078 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:46.405002 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428aa488-19d8-4de6-aff4-ccaf77d5a162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:46.407267 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-79293897-886f-4c72-b9c8-c5f3f218b078 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:46.407777 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428aa488-19d8-4de6-aff4-ccaf77d5a162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:46.408198 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428aa488-19d8-4de6-aff4-ccaf77d5a162 admin admin] PATH_INFO: `/v3/users/56b6a40596ce4c0fab0cf7def22fbc54` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:46.412600 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428aa488-19d8-4de6-aff4-ccaf77d5a162 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56b6a40596ce4c0fab0cf7def22fbc54)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:46.417502 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: WARNING py.warnings [None req-428aa488-19d8-4de6-aff4-ccaf77d5a162 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 03 16:15:46.417502 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: warnings.warn(msg) Jun 03 16:15:46.417502 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: Jun 03 16:15:46.424606 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428aa488-19d8-4de6-aff4-ccaf77d5a162 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:46.540494 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:46.669400 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-428aa488-19d8-4de6-aff4-ccaf77d5a162 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56b6a40596ce4c0fab0cf7def22fbc54'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:46.682822 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79293897-886f-4c72-b9c8-c5f3f218b078 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:46.687363 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-428aa488-19d8-4de6-aff4-ccaf77d5a162 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56b6a40596ce4c0fab0cf7def22fbc54'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:46.698378 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e71df2-7f62-46c9-9e83-8947a7ca0fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:46.701225 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e71df2-7f62-46c9-9e83-8947a7ca0fd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:46.701547 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e71df2-7f62-46c9-9e83-8947a7ca0fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:46.701888 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e71df2-7f62-46c9-9e83-8947a7ca0fd9 admin admin] PATH_INFO: `/v3/users/866c56c184f14dde80e90670dc3e758f` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:46.704259 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e71df2-7f62-46c9-9e83-8947a7ca0fd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=866c56c184f14dde80e90670dc3e758f)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:46.708664 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e71df2-7f62-46c9-9e83-8947a7ca0fd9 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:46.745547 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-428aa488-19d8-4de6-aff4-ccaf77d5a162 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56b6a40596ce4c0fab0cf7def22fbc54'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:46.769521 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-428aa488-19d8-4de6-aff4-ccaf77d5a162 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56b6a40596ce4c0fab0cf7def22fbc54'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:46.772347 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-428aa488-19d8-4de6-aff4-ccaf77d5a162 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56b6a40596ce4c0fab0cf7def22fbc54'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:46.805921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 841/1659] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:15:46 2022] DELETE /identity/v3/users/56b6a40596ce4c0fab0cf7def22fbc54 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:15:46.813313 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c888c-3f0d-4886-8247-53b397073598 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:46.813658 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c888c-3f0d-4886-8247-53b397073598 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:46.813972 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c888c-3f0d-4886-8247-53b397073598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:46.870710 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-373c888c-3f0d-4886-8247-53b397073598 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:46.872068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-77e71df2-7f62-46c9-9e83-8947a7ca0fd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '866c56c184f14dde80e90670dc3e758f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:46.872528 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-373c888c-3f0d-4886-8247-53b397073598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:46.906959 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-77e71df2-7f62-46c9-9e83-8947a7ca0fd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '866c56c184f14dde80e90670dc3e758f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:46.928823 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-77e71df2-7f62-46c9-9e83-8947a7ca0fd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '866c56c184f14dde80e90670dc3e758f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:46.931350 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-77e71df2-7f62-46c9-9e83-8947a7ca0fd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '866c56c184f14dde80e90670dc3e758f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:46.947644 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-77e71df2-7f62-46c9-9e83-8947a7ca0fd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '866c56c184f14dde80e90670dc3e758f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:46.975656 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 819/1660] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:15:46 2022] DELETE /identity/v3/users/866c56c184f14dde80e90670dc3e758f => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:15:46.978193 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06838937-b9dc-4b76-995b-824cc7377dfc None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:46.983072 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06838937-b9dc-4b76-995b-824cc7377dfc None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:46.983836 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06838937-b9dc-4b76-995b-824cc7377dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:47.054300 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-06838937-b9dc-4b76-995b-824cc7377dfc None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:47.057002 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 842/1661] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:15:47.057732 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-06838937-b9dc-4b76-995b-824cc7377dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:47.059425 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f108621-24e9-4164-a9dc-9be24c879750 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:47.059730 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f108621-24e9-4164-a9dc-9be24c879750 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:47.060019 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f108621-24e9-4164-a9dc-9be24c879750 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:47.062446 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 843/1662] 10.222.0.41 () {54 vars in 960 bytes} [Fri Jun 3 16:15:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:15:47.070907 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81267cc4-2bfa-4422-b14c-b6b883376b44 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:47.071826 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81267cc4-2bfa-4422-b14c-b6b883376b44 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:47.072316 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81267cc4-2bfa-4422-b14c-b6b883376b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:47.127049 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-81267cc4-2bfa-4422-b14c-b6b883376b44 None None] MFA Rules not processed for user `be901786f8934d559e0bac5d622f97cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:47.132441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-81267cc4-2bfa-4422-b14c-b6b883376b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:47.140670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:47.213294 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 820/1663] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:47.214123 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06838937-b9dc-4b76-995b-824cc7377dfc None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:47.222367 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-06838937-b9dc-4b76-995b-824cc7377dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:47.289491 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 844/1664] 10.222.0.41 () {58 vars in 1057 bytes} [Fri Jun 3 16:15:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:47.296510 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81267cc4-2bfa-4422-b14c-b6b883376b44 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:47.310431 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81267cc4-2bfa-4422-b14c-b6b883376b44 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:47.317791 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7560a4fb-0835-4ee1-a567-0f916a23d518 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be901786f8934d559e0bac5d622f97cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:47.318244 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560a4fb-0835-4ee1-a567-0f916a23d518 service placement] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:47.318640 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560a4fb-0835-4ee1-a567-0f916a23d518 service placement] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:47.319026 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560a4fb-0835-4ee1-a567-0f916a23d518 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:47.321356 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-7560a4fb-0835-4ee1-a567-0f916a23d518 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:47.340846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7560a4fb-0835-4ee1-a567-0f916a23d518 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:47.343852 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7560a4fb-0835-4ee1-a567-0f916a23d518 service placement] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:47.362373 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 845/1665] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:15:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:15:47.365091 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7560a4fb-0835-4ee1-a567-0f916a23d518 service placement] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:47.367433 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-7560a4fb-0835-4ee1-a567-0f916a23d518 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:47.394785 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06838937-b9dc-4b76-995b-824cc7377dfc None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:47.398626 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:47.408662 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28d834f-a6d5-47df-a066-c9b8abe89ed0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:47.410162 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d834f-a6d5-47df-a066-c9b8abe89ed0 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:47.410671 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d834f-a6d5-47df-a066-c9b8abe89ed0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:47.411157 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d834f-a6d5-47df-a066-c9b8abe89ed0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:47.432009 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28d834f-a6d5-47df-a066-c9b8abe89ed0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:47.433901 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28d834f-a6d5-47df-a066-c9b8abe89ed0 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:47.464390 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 821/1666] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:15:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:15:47.494897 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7560a4fb-0835-4ee1-a567-0f916a23d518 service placement] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:47.500138 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49123a84-427e-45d6-b11b-c6eedd3171b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:47.500830 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49123a84-427e-45d6-b11b-c6eedd3171b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:47.501118 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49123a84-427e-45d6-b11b-c6eedd3171b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:47.501398 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49123a84-427e-45d6-b11b-c6eedd3171b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:47.514581 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49123a84-427e-45d6-b11b-c6eedd3171b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:47.516188 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49123a84-427e-45d6-b11b-c6eedd3171b6 service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:47.533791 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 846/1667] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:15:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:15:47.659756 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3abbe-d32c-4af5-8d12-ff0100623270 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:47.660049 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3abbe-d32c-4af5-8d12-ff0100623270 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:47.660336 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3abbe-d32c-4af5-8d12-ff0100623270 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:47.661885 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 822/1668] 10.222.0.41 () {54 vars in 961 bytes} [Fri Jun 3 16:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:15:47.671859 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98925393-4f08-4883-b8a8-8277f8bb172c None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:47.672394 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98925393-4f08-4883-b8a8-8277f8bb172c None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:47.672394 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98925393-4f08-4883-b8a8-8277f8bb172c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:47.693740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-98925393-4f08-4883-b8a8-8277f8bb172c None None] MFA Rules not processed for user `2fd08ea39ee94b92ac6763608e69f158`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:47.694984 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-98925393-4f08-4883-b8a8-8277f8bb172c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:47.802922 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 847/1669] 10.222.0.41 () {58 vars in 1058 bytes} [Fri Jun 3 16:15:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:47.812552 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a639d5-a3e3-443d-8c43-01409c78a383 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:47.813066 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a639d5-a3e3-443d-8c43-01409c78a383 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:47.814115 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a639d5-a3e3-443d-8c43-01409c78a383 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:47.815902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 823/1670] 10.222.0.41 () {54 vars in 912 bytes} [Fri Jun 3 16:15:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:15:47.822874 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f1224-6af1-44af-ac29-bdf5871eb8cf None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:47.823171 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f1224-6af1-44af-ac29-bdf5871eb8cf None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:47.823776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f1224-6af1-44af-ac29-bdf5871eb8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:47.849845 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-da6f1224-6af1-44af-ac29-bdf5871eb8cf None None] MFA Rules not processed for user `2fd08ea39ee94b92ac6763608e69f158`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:47.851279 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-da6f1224-6af1-44af-ac29-bdf5871eb8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:47.863568 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 848/1671] 10.222.0.41 () {58 vars in 1009 bytes} [Fri Jun 3 16:15:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:47.890542 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a639d5-a3e3-443d-8c43-01409c78a383 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:47.927948 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a639d5-a3e3-443d-8c43-01409c78a383 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:47.950281 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a0f6b3-b43f-4f8e-9631-6d7363d7937d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:47.951427 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0f6b3-b43f-4f8e-9631-6d7363d7937d service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:47.952930 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0f6b3-b43f-4f8e-9631-6d7363d7937d service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:47.953407 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0f6b3-b43f-4f8e-9631-6d7363d7937d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:48.001518 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a0f6b3-b43f-4f8e-9631-6d7363d7937d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:48.004020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a0f6b3-b43f-4f8e-9631-6d7363d7937d service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:48.045904 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 824/1672] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:15:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:15:48.803132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f1224-6af1-44af-ac29-bdf5871eb8cf None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:48.805647 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-da6f1224-6af1-44af-ac29-bdf5871eb8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:48.831161 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f1224-6af1-44af-ac29-bdf5871eb8cf None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:48.840877 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef94abf7-9c57-415e-a274-80e4edf27fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:48.842045 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94abf7-9c57-415e-a274-80e4edf27fdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:48.842774 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94abf7-9c57-415e-a274-80e4edf27fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:48.843257 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94abf7-9c57-415e-a274-80e4edf27fdd admin admin] PATH_INFO: `/v3/projects/d648cc83af4a453b8f1472d9a1c58464` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:48.845569 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef94abf7-9c57-415e-a274-80e4edf27fdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d648cc83af4a453b8f1472d9a1c58464)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:48.850596 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef94abf7-9c57-415e-a274-80e4edf27fdd admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:48.960073 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-ef94abf7-9c57-415e-a274-80e4edf27fdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd648cc83af4a453b8f1472d9a1c58464', 'request_id': 'req-ef94abf7-9c57-415e-a274-80e4edf27fdd'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:48.976425 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-ef94abf7-9c57-415e-a274-80e4edf27fdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd648cc83af4a453b8f1472d9a1c58464', 'request_id': 'req-ef94abf7-9c57-415e-a274-80e4edf27fdd'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:48.993943 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-ef94abf7-9c57-415e-a274-80e4edf27fdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd648cc83af4a453b8f1472d9a1c58464', 'request_id': 'req-ef94abf7-9c57-415e-a274-80e4edf27fdd'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:49.031523 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-ef94abf7-9c57-415e-a274-80e4edf27fdd admin admin] The token cache is being invalidate because project d648cc83af4a453b8f1472d9a1c58464 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:15:49.032554 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-ef94abf7-9c57-415e-a274-80e4edf27fdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:49.034632 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-ef94abf7-9c57-415e-a274-80e4edf27fdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:49.042248 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 849/1673] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:15:48 2022] DELETE /identity/v3/projects/d648cc83af4a453b8f1472d9a1c58464 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:15:49.044315 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3554e-6947-4b97-99cc-5ed76abd819a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:49.044805 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3554e-6947-4b97-99cc-5ed76abd819a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:49.044879 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3554e-6947-4b97-99cc-5ed76abd819a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:49.098857 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-69f3554e-6947-4b97-99cc-5ed76abd819a None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:49.103535 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-69f3554e-6947-4b97-99cc-5ed76abd819a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:49.235333 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef94abf7-9c57-415e-a274-80e4edf27fdd admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:49.237551 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-ef94abf7-9c57-415e-a274-80e4edf27fdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:49.329248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 825/1674] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:49.330643 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0c901-199f-4735-bae8-d77b50215e2f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:49.330941 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0c901-199f-4735-bae8-d77b50215e2f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:49.331223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0c901-199f-4735-bae8-d77b50215e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:49.362485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:49.422780 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-2ae0c901-199f-4735-bae8-d77b50215e2f None None] MFA Rules not processed for user `94f91ae071b24e31a8ef8d50933ba4fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:49.424436 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2ae0c901-199f-4735-bae8-d77b50215e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:49.471503 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef94abf7-9c57-415e-a274-80e4edf27fdd admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:49.484515 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d157e5c1-caf2-4683-8845-b0dafa5980bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:49.485413 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d157e5c1-caf2-4683-8845-b0dafa5980bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:49.486258 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d157e5c1-caf2-4683-8845-b0dafa5980bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:49.486258 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d157e5c1-caf2-4683-8845-b0dafa5980bc admin admin] PATH_INFO: `/v3/projects/34a04db02b7148da89f56b89dc1921eb` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:49.490527 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d157e5c1-caf2-4683-8845-b0dafa5980bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=34a04db02b7148da89f56b89dc1921eb)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:49.492577 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d157e5c1-caf2-4683-8845-b0dafa5980bc admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:49.569102 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:49.584830 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 826/1675] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:49.584830 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921dfea-6484-4649-a757-e4c6deaf495d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:49.584830 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921dfea-6484-4649-a757-e4c6deaf495d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:49.584830 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921dfea-6484-4649-a757-e4c6deaf495d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:49.609199 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-7921dfea-6484-4649-a757-e4c6deaf495d None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:49.610504 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7921dfea-6484-4649-a757-e4c6deaf495d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:49.621023 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-d157e5c1-caf2-4683-8845-b0dafa5980bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34a04db02b7148da89f56b89dc1921eb', 'request_id': 'req-d157e5c1-caf2-4683-8845-b0dafa5980bc'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:49.635428 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-d157e5c1-caf2-4683-8845-b0dafa5980bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34a04db02b7148da89f56b89dc1921eb', 'request_id': 'req-d157e5c1-caf2-4683-8845-b0dafa5980bc'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:49.650290 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-d157e5c1-caf2-4683-8845-b0dafa5980bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34a04db02b7148da89f56b89dc1921eb', 'request_id': 'req-d157e5c1-caf2-4683-8845-b0dafa5980bc'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:49.673930 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-d157e5c1-caf2-4683-8845-b0dafa5980bc admin admin] The token cache is being invalidate because project 34a04db02b7148da89f56b89dc1921eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:15:49.674271 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-d157e5c1-caf2-4683-8845-b0dafa5980bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:49.674889 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-d157e5c1-caf2-4683-8845-b0dafa5980bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:49.686367 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 850/1676] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:15:49 2022] DELETE /identity/v3/projects/34a04db02b7148da89f56b89dc1921eb => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:15:49.687333 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d157e5c1-caf2-4683-8845-b0dafa5980bc admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:49.694592 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-d157e5c1-caf2-4683-8845-b0dafa5980bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:49.740436 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 827/1677] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:49.741805 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921dfea-6484-4649-a757-e4c6deaf495d None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:49.750450 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7921dfea-6484-4649-a757-e4c6deaf495d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:49.782681 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921dfea-6484-4649-a757-e4c6deaf495d None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:49.793872 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb57ec3-1794-4fa4-918a-e2e9a655e872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:49.794785 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb57ec3-1794-4fa4-918a-e2e9a655e872 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:49.795073 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb57ec3-1794-4fa4-918a-e2e9a655e872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:49.795388 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb57ec3-1794-4fa4-918a-e2e9a655e872 admin admin] PATH_INFO: `/v3/users/829fa2e7304c4a33920a7194a81cc9e6` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:49.797472 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb57ec3-1794-4fa4-918a-e2e9a655e872 admin admin] RBAC: Authorizing `identity:delete_user(user_id=829fa2e7304c4a33920a7194a81cc9e6)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:49.802044 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb57ec3-1794-4fa4-918a-e2e9a655e872 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:49.894425 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d157e5c1-caf2-4683-8845-b0dafa5980bc admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:49.907670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-dfb57ec3-1794-4fa4-918a-e2e9a655e872 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '829fa2e7304c4a33920a7194a81cc9e6'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:49.912071 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9947d-4188-4b69-a69d-841fa85c94d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:49.912819 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9947d-4188-4b69-a69d-841fa85c94d7 service nova] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:49.913112 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9947d-4188-4b69-a69d-841fa85c94d7 service nova] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:49.915358 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9947d-4188-4b69-a69d-841fa85c94d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:49.918927 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-1ed9947d-4188-4b69-a69d-841fa85c94d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:49.922500 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-dfb57ec3-1794-4fa4-918a-e2e9a655e872 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '829fa2e7304c4a33920a7194a81cc9e6'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:49.951068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-dfb57ec3-1794-4fa4-918a-e2e9a655e872 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '829fa2e7304c4a33920a7194a81cc9e6'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:49.952833 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-dfb57ec3-1794-4fa4-918a-e2e9a655e872 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '829fa2e7304c4a33920a7194a81cc9e6'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:49.969468 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-dfb57ec3-1794-4fa4-918a-e2e9a655e872 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '829fa2e7304c4a33920a7194a81cc9e6'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:49.985717 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 828/1678] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:15:49 2022] DELETE /identity/v3/users/829fa2e7304c4a33920a7194a81cc9e6 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:15:49.987828 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb57ec3-1794-4fa4-918a-e2e9a655e872 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:49.990595 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-dfb57ec3-1794-4fa4-918a-e2e9a655e872 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:50.103754 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:50.112327 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb57ec3-1794-4fa4-918a-e2e9a655e872 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:50.119016 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1f623d-6fa6-4837-ab2e-0e599242d943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:50.120586 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f623d-6fa6-4837-ab2e-0e599242d943 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:50.120921 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f623d-6fa6-4837-ab2e-0e599242d943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:50.121363 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f623d-6fa6-4837-ab2e-0e599242d943 admin admin] PATH_INFO: `/v3/projects/3a0bf140eda04cedac1bba0d46246b0a` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:50.123132 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1f623d-6fa6-4837-ab2e-0e599242d943 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a0bf140eda04cedac1bba0d46246b0a)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:50.124403 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: WARNING py.warnings [None req-9c1f623d-6fa6-4837-ab2e-0e599242d943 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 03 16:15:50.124403 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: warnings.warn(msg) Jun 03 16:15:50.124403 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: Jun 03 16:15:50.124815 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed9947d-4188-4b69-a69d-841fa85c94d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:50.125606 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1f623d-6fa6-4837-ab2e-0e599242d943 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:50.126438 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed9947d-4188-4b69-a69d-841fa85c94d7 service nova] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:50.149496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:50.156617 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 851/1679] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:15:49 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:15:50.163516 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9947d-4188-4b69-a69d-841fa85c94d7 service nova] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:50.202408 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9947d-4188-4b69-a69d-841fa85c94d7 service nova] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:50.265510 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9c1f623d-6fa6-4837-ab2e-0e599242d943 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a0bf140eda04cedac1bba0d46246b0a', 'request_id': 'req-9c1f623d-6fa6-4837-ab2e-0e599242d943'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:50.321711 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ebf28b-3f01-4d59-b6fe-5d8666e67e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:50.322790 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ebf28b-3f01-4d59-b6fe-5d8666e67e5d admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:50.323587 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9c1f623d-6fa6-4837-ab2e-0e599242d943 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a0bf140eda04cedac1bba0d46246b0a', 'request_id': 'req-9c1f623d-6fa6-4837-ab2e-0e599242d943'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:50.331919 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ebf28b-3f01-4d59-b6fe-5d8666e67e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:50.332686 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ebf28b-3f01-4d59-b6fe-5d8666e67e5d admin admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:50.333705 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ebf28b-3f01-4d59-b6fe-5d8666e67e5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:15:50.334629 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ebf28b-3f01-4d59-b6fe-5d8666e67e5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:50.336500 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ebf28b-3f01-4d59-b6fe-5d8666e67e5d admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:50.346828 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9c1f623d-6fa6-4837-ab2e-0e599242d943 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a0bf140eda04cedac1bba0d46246b0a', 'request_id': 'req-9c1f623d-6fa6-4837-ab2e-0e599242d943'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:50.353752 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 852/1680] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:15:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:15:50.359468 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ebf28b-3f01-4d59-b6fe-5d8666e67e5d admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:50.362339 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-89ebf28b-3f01-4d59-b6fe-5d8666e67e5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:50.390112 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9c1f623d-6fa6-4837-ab2e-0e599242d943 admin admin] The token cache is being invalidate because project 3a0bf140eda04cedac1bba0d46246b0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:15:50.390891 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9c1f623d-6fa6-4837-ab2e-0e599242d943 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:50.392010 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9c1f623d-6fa6-4837-ab2e-0e599242d943 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:50.393579 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ebf28b-3f01-4d59-b6fe-5d8666e67e5d admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:50.395102 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 829/1681] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:15:49 2022] DELETE /identity/v3/projects/3a0bf140eda04cedac1bba0d46246b0a => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:15:50.401125 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1f623d-6fa6-4837-ab2e-0e599242d943 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:50.403897 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9c1f623d-6fa6-4837-ab2e-0e599242d943 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:50.409216 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf7784-05bb-46bc-978a-173b66dc44fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:50.410616 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf7784-05bb-46bc-978a-173b66dc44fe admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:50.410616 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf7784-05bb-46bc-978a-173b66dc44fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:50.410616 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf7784-05bb-46bc-978a-173b66dc44fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:50.412655 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bf7784-05bb-46bc-978a-173b66dc44fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:50.417130 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bf7784-05bb-46bc-978a-173b66dc44fe admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:50.439045 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1f623d-6fa6-4837-ab2e-0e599242d943 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:50.454599 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9300a8b-2d80-4f63-9e03-a694865b1843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:50.455538 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9300a8b-2d80-4f63-9e03-a694865b1843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:50.456270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9300a8b-2d80-4f63-9e03-a694865b1843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:50.456566 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9300a8b-2d80-4f63-9e03-a694865b1843 admin admin] PATH_INFO: `/v3/users/bbc35976c35f42eba4e29f69c3f9f8b3` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:50.461172 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9300a8b-2d80-4f63-9e03-a694865b1843 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbc35976c35f42eba4e29f69c3f9f8b3)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:50.468507 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9300a8b-2d80-4f63-9e03-a694865b1843 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:50.482210 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 853/1682] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:15:50 2022] POST /identity/v3/projects => generated 365 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:15:50.483849 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf7784-05bb-46bc-978a-173b66dc44fe admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:50.488162 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-72bf7784-05bb-46bc-978a-173b66dc44fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:50.567919 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b9300a8b-2d80-4f63-9e03-a694865b1843 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbc35976c35f42eba4e29f69c3f9f8b3'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:50.587409 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b9300a8b-2d80-4f63-9e03-a694865b1843 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbc35976c35f42eba4e29f69c3f9f8b3'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:50.612963 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b9300a8b-2d80-4f63-9e03-a694865b1843 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbc35976c35f42eba4e29f69c3f9f8b3'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:50.616759 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b9300a8b-2d80-4f63-9e03-a694865b1843 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbc35976c35f42eba4e29f69c3f9f8b3'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:50.632766 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b9300a8b-2d80-4f63-9e03-a694865b1843 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbc35976c35f42eba4e29f69c3f9f8b3'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:50.645742 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 830/1683] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:15:50 2022] DELETE /identity/v3/users/bbc35976c35f42eba4e29f69c3f9f8b3 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:15:50.654765 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf7784-05bb-46bc-978a-173b66dc44fe admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:50.677780 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e71cac-b86f-409f-bca2-8afe6191cb66 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:50.678122 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e71cac-b86f-409f-bca2-8afe6191cb66 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:50.678342 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e71cac-b86f-409f-bca2-8afe6191cb66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:50.718803 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c3e71cac-b86f-409f-bca2-8afe6191cb66 None None] MFA Rules not processed for user `875140cf51d9410da45f8c3805b26204`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:50.723254 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c3e71cac-b86f-409f-bca2-8afe6191cb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:50.800211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d8216-12ba-4d29-b820-eb91a7e76583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:50.801374 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d8216-12ba-4d29-b820-eb91a7e76583 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:50.801841 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d8216-12ba-4d29-b820-eb91a7e76583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:50.802349 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d8216-12ba-4d29-b820-eb91a7e76583 admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:50.803226 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781d8216-12ba-4d29-b820-eb91a7e76583 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:50.806104 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781d8216-12ba-4d29-b820-eb91a7e76583 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:50.849705 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 854/1684] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:15:50 2022] POST /identity/v3/users => generated 283 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:15:50.853659 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d8216-12ba-4d29-b820-eb91a7e76583 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:50.856665 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 831/1685] 10.222.0.41 () {58 vars in 1009 bytes} [Fri Jun 3 16:15:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:15:50.856977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-781d8216-12ba-4d29-b820-eb91a7e76583 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:50.873463 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:50.885997 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d8216-12ba-4d29-b820-eb91a7e76583 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:50.914314 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb3da5c-3ea3-4106-b04e-07133e1d824c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:50.915584 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb3da5c-3ea3-4106-b04e-07133e1d824c admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:50.916016 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb3da5c-3ea3-4106-b04e-07133e1d824c admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:50.916485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb3da5c-3ea3-4106-b04e-07133e1d824c admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:50.917453 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb3da5c-3ea3-4106-b04e-07133e1d824c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:50.919217 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb3da5c-3ea3-4106-b04e-07133e1d824c admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:50.955891 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 855/1686] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:15:50.957388 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e71cac-b86f-409f-bca2-8afe6191cb66 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:50.975514 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e71cac-b86f-409f-bca2-8afe6191cb66 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:50.988037 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac737a11-fe4e-4d89-a70c-fc745b05cc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:50.989098 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac737a11-fe4e-4d89-a70c-fc745b05cc7e admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:50.989098 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac737a11-fe4e-4d89-a70c-fc745b05cc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:50.989331 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac737a11-fe4e-4d89-a70c-fc745b05cc7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:50.990336 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac737a11-fe4e-4d89-a70c-fc745b05cc7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:50.991902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac737a11-fe4e-4d89-a70c-fc745b05cc7e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:51.020044 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 832/1687] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:15:51.021043 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb3da5c-3ea3-4106-b04e-07133e1d824c admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:51.050959 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb3da5c-3ea3-4106-b04e-07133e1d824c admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:51.062530 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc6a08e-abe8-4bcb-9c39-3031aa34af79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:51.064775 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc6a08e-abe8-4bcb-9c39-3031aa34af79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:51.066336 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc6a08e-abe8-4bcb-9c39-3031aa34af79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:51.066936 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc6a08e-abe8-4bcb-9c39-3031aa34af79 admin admin] PATH_INFO: `/v3/projects/3d3fc547ac324886a3cfb147650b4965/users/ae72489e4aaa4253a2204eb1f66121e9/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:51.126385 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc6a08e-abe8-4bcb-9c39-3031aa34af79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d3fc547ac324886a3cfb147650b4965, user_id=ae72489e4aaa4253a2204eb1f66121e9, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:51.129409 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc6a08e-abe8-4bcb-9c39-3031aa34af79 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:51.159186 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 856/1688] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:15:51 2022] PUT /identity/v3/projects/3d3fc547ac324886a3cfb147650b4965/users/ae72489e4aaa4253a2204eb1f66121e9/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:15:51.170470 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac737a11-fe4e-4d89-a70c-fc745b05cc7e admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:51.172393 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ac737a11-fe4e-4d89-a70c-fc745b05cc7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:51.173196 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5699ee-22c8-459f-b24a-d3ea720081e6 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:51.173591 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5699ee-22c8-459f-b24a-d3ea720081e6 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:51.174065 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5699ee-22c8-459f-b24a-d3ea720081e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:51.194039 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:51.218329 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-4b5699ee-22c8-459f-b24a-d3ea720081e6 None None] MFA Rules not processed for user `ae72489e4aaa4253a2204eb1f66121e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:51.219897 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-4b5699ee-22c8-459f-b24a-d3ea720081e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:51.353631 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac737a11-fe4e-4d89-a70c-fc745b05cc7e admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:51.388542 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 857/1689] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:51 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:51.393075 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5699ee-22c8-459f-b24a-d3ea720081e6 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:51.536785 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5699ee-22c8-459f-b24a-d3ea720081e6 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:51.539640 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a98be2-6ce6-4968-9652-19ff9e9d5eff service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5f513b781a4a73b443cded13359eda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:51.540605 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a98be2-6ce6-4968-9652-19ff9e9d5eff service swift] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:51.541921 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a98be2-6ce6-4968-9652-19ff9e9d5eff service swift] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:51.543991 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833904c0-ffe8-482c-8d16-42204e34aa6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:51.544424 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a98be2-6ce6-4968-9652-19ff9e9d5eff service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:51.544894 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833904c0-ffe8-482c-8d16-42204e34aa6d admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:51.545281 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833904c0-ffe8-482c-8d16-42204e34aa6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:51.545659 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833904c0-ffe8-482c-8d16-42204e34aa6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:51.547003 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833904c0-ffe8-482c-8d16-42204e34aa6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:51.549823 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833904c0-ffe8-482c-8d16-42204e34aa6d admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:51.682645 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 858/1690] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:15:51 2022] POST /identity/v3/projects => generated 363 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:15:51.685035 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68639f79-a426-4db7-a99a-a5bb68f1ee54 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:51.685836 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68639f79-a426-4db7-a99a-a5bb68f1ee54 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:51.686632 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68639f79-a426-4db7-a99a-a5bb68f1ee54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:51.716806 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a98be2-6ce6-4968-9652-19ff9e9d5eff service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:51.719675 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a98be2-6ce6-4968-9652-19ff9e9d5eff service swift] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:51.740498 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-68639f79-a426-4db7-a99a-a5bb68f1ee54 None None] MFA Rules not processed for user `ef7c1e449b864b50b3c71c39f19968a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:51.742693 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-68639f79-a426-4db7-a99a-a5bb68f1ee54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:51.743022 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 833/1691] 10.222.0.41 () {60 vars in 1384 bytes} [Fri Jun 3 16:15:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 16:15:51.746434 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a98be2-6ce6-4968-9652-19ff9e9d5eff service swift] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:51.771381 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a98be2-6ce6-4968-9652-19ff9e9d5eff service swift] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:51.809309 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:51.872893 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 859/1692] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:51.889014 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68639f79-a426-4db7-a99a-a5bb68f1ee54 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:51.893046 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-68639f79-a426-4db7-a99a-a5bb68f1ee54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:52.116045 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:52.175914 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1fbdaf-7e60-4fce-999a-739f460b0622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:52.176784 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1fbdaf-7e60-4fce-999a-739f460b0622 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:52.176881 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1fbdaf-7e60-4fce-999a-739f460b0622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:52.177150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1fbdaf-7e60-4fce-999a-739f460b0622 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:52.177870 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1fbdaf-7e60-4fce-999a-739f460b0622 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:52.180009 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1fbdaf-7e60-4fce-999a-739f460b0622 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:52.252711 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 834/1693] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:15:51 2022] POST /identity/v3/users => generated 288 bytes in 509 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:15:52.254271 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1fbdaf-7e60-4fce-999a-739f460b0622 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:52.287254 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1fbdaf-7e60-4fce-999a-739f460b0622 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:52.294111 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68639f79-a426-4db7-a99a-a5bb68f1ee54 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:52.317943 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31bc23-75ef-4a62-b4eb-b186c77c9014 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:52.319716 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28aa0a7-0b41-4dfb-aece-36b1a99ea3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:52.320416 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28aa0a7-0b41-4dfb-aece-36b1a99ea3da admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:52.320711 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28aa0a7-0b41-4dfb-aece-36b1a99ea3da admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:52.320998 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28aa0a7-0b41-4dfb-aece-36b1a99ea3da admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:52.322354 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28aa0a7-0b41-4dfb-aece-36b1a99ea3da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:52.323103 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31bc23-75ef-4a62-b4eb-b186c77c9014 service nova] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:52.324032 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28aa0a7-0b41-4dfb-aece-36b1a99ea3da admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:52.325282 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31bc23-75ef-4a62-b4eb-b186c77c9014 service nova] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:52.326958 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31bc23-75ef-4a62-b4eb-b186c77c9014 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:52.366514 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 835/1694] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:15:52.367545 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28aa0a7-0b41-4dfb-aece-36b1a99ea3da admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:52.373779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31bc23-75ef-4a62-b4eb-b186c77c9014 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:52.383320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31bc23-75ef-4a62-b4eb-b186c77c9014 service nova] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:52.394369 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28aa0a7-0b41-4dfb-aece-36b1a99ea3da admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:52.407377 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4b7c2-061b-457b-8c24-c4f239c8197b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:52.408300 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4b7c2-061b-457b-8c24-c4f239c8197b admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:52.408596 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4b7c2-061b-457b-8c24-c4f239c8197b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:52.408867 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4b7c2-061b-457b-8c24-c4f239c8197b admin admin] PATH_INFO: `/v3/projects/97daef81233647f48d7a1c4ff67cc313/users/608c2aca5d514c758ca81f026d642b00/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:52.444838 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 860/1695] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:15:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:15:52.452980 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d4b7c2-061b-457b-8c24-c4f239c8197b admin admin] RBAC: Authorizing `identity:create_grant(project_id=97daef81233647f48d7a1c4ff67cc313, user_id=608c2aca5d514c758ca81f026d642b00, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:52.456044 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d4b7c2-061b-457b-8c24-c4f239c8197b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:52.491253 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 836/1696] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:15:52 2022] PUT /identity/v3/projects/97daef81233647f48d7a1c4ff67cc313/users/608c2aca5d514c758ca81f026d642b00/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:15:52.494924 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31bc23-75ef-4a62-b4eb-b186c77c9014 service nova] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:52.515048 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31bc23-75ef-4a62-b4eb-b186c77c9014 service nova] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:52.798642 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834c030-a4aa-4e09-b823-80cb2b80f174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:52.804682 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834c030-a4aa-4e09-b823-80cb2b80f174 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:52.804682 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834c030-a4aa-4e09-b823-80cb2b80f174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:52.804898 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834c030-a4aa-4e09-b823-80cb2b80f174 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:52.807221 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834c030-a4aa-4e09-b823-80cb2b80f174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:52.826044 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834c030-a4aa-4e09-b823-80cb2b80f174 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:52.872328 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 861/1697] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:15:52.878453 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4b7c2-061b-457b-8c24-c4f239c8197b admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:52.897204 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4b7c2-061b-457b-8c24-c4f239c8197b admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:52.909011 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1befbe6-e514-4f57-9132-a11ef0a35a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:52.910077 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1befbe6-e514-4f57-9132-a11ef0a35a74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:52.910411 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1befbe6-e514-4f57-9132-a11ef0a35a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:52.910697 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1befbe6-e514-4f57-9132-a11ef0a35a74 admin admin] PATH_INFO: `/v3/projects/97daef81233647f48d7a1c4ff67cc313/users/608c2aca5d514c758ca81f026d642b00/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:52.915576 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1befbe6-e514-4f57-9132-a11ef0a35a74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97daef81233647f48d7a1c4ff67cc313, user_id=608c2aca5d514c758ca81f026d642b00, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:52.921650 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1befbe6-e514-4f57-9132-a11ef0a35a74 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:52.951844 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 837/1698] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:15:52 2022] PUT /identity/v3/projects/97daef81233647f48d7a1c4ff67cc313/users/608c2aca5d514c758ca81f026d642b00/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:15:52.963447 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786e915-5c1e-4e9c-891e-9ff7ecdd1242 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:52.963743 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786e915-5c1e-4e9c-891e-9ff7ecdd1242 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:52.964031 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786e915-5c1e-4e9c-891e-9ff7ecdd1242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:53.055619 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-d786e915-5c1e-4e9c-891e-9ff7ecdd1242 None None] MFA Rules not processed for user `608c2aca5d514c758ca81f026d642b00`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:53.056893 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-d786e915-5c1e-4e9c-891e-9ff7ecdd1242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:53.301808 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1befbe6-e514-4f57-9132-a11ef0a35a74 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:53.303605 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b1befbe6-e514-4f57-9132-a11ef0a35a74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:53.370253 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:53.548573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 862/1699] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:52 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 587 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:53.564574 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d786e915-5c1e-4e9c-891e-9ff7ecdd1242 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:53.569844 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-d786e915-5c1e-4e9c-891e-9ff7ecdd1242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:53.621595 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1befbe6-e514-4f57-9132-a11ef0a35a74 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:53.652463 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8509b1f-d462-4a3e-b94f-72fcdc28ba22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:53.653763 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8509b1f-d462-4a3e-b94f-72fcdc28ba22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:53.654051 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8509b1f-d462-4a3e-b94f-72fcdc28ba22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:53.654329 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8509b1f-d462-4a3e-b94f-72fcdc28ba22 admin admin] PATH_INFO: `/v3/projects/fe9066ff903c4d91b49e294a4b9c9817` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:53.659041 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8509b1f-d462-4a3e-b94f-72fcdc28ba22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe9066ff903c4d91b49e294a4b9c9817)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:53.663018 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8509b1f-d462-4a3e-b94f-72fcdc28ba22 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:53.833300 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d8509b1f-d462-4a3e-b94f-72fcdc28ba22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe9066ff903c4d91b49e294a4b9c9817', 'request_id': 'req-d8509b1f-d462-4a3e-b94f-72fcdc28ba22'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:53.849958 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d786e915-5c1e-4e9c-891e-9ff7ecdd1242 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:53.863870 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d8509b1f-d462-4a3e-b94f-72fcdc28ba22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe9066ff903c4d91b49e294a4b9c9817', 'request_id': 'req-d8509b1f-d462-4a3e-b94f-72fcdc28ba22'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:53.890133 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d8509b1f-d462-4a3e-b94f-72fcdc28ba22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe9066ff903c4d91b49e294a4b9c9817', 'request_id': 'req-d8509b1f-d462-4a3e-b94f-72fcdc28ba22'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:53.939625 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d8509b1f-d462-4a3e-b94f-72fcdc28ba22 admin admin] The token cache is being invalidate because project fe9066ff903c4d91b49e294a4b9c9817 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:15:53.939918 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d8509b1f-d462-4a3e-b94f-72fcdc28ba22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:53.941932 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d8509b1f-d462-4a3e-b94f-72fcdc28ba22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:53.946019 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 838/1700] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:15:53 2022] DELETE /identity/v3/projects/fe9066ff903c4d91b49e294a4b9c9817 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:15:53.950850 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e2406-52b2-40fc-a060-d91f5b7bc258 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:53.951146 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e2406-52b2-40fc-a060-d91f5b7bc258 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:53.951462 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e2406-52b2-40fc-a060-d91f5b7bc258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:54.032310 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-474e2406-52b2-40fc-a060-d91f5b7bc258 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:54.033568 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-474e2406-52b2-40fc-a060-d91f5b7bc258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:54.100442 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20eb827-9a87-43e0-892c-b1afde575c3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:54.103895 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20eb827-9a87-43e0-892c-b1afde575c3f service cinder] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:54.106396 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20eb827-9a87-43e0-892c-b1afde575c3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:54.107405 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20eb827-9a87-43e0-892c-b1afde575c3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:54.114818 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-c20eb827-9a87-43e0-892c-b1afde575c3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:54.129493 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:54.251535 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 839/1701] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:54.254118 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a9941-eec4-47ba-8884-3c609bd0e297 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:54.254563 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a9941-eec4-47ba-8884-3c609bd0e297 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:54.254977 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a9941-eec4-47ba-8884-3c609bd0e297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:54.290774 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-270a9941-eec4-47ba-8884-3c609bd0e297 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:54.293004 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-270a9941-eec4-47ba-8884-3c609bd0e297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:54.335656 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 840/1702] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:54.337537 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a9941-eec4-47ba-8884-3c609bd0e297 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:54.343903 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-270a9941-eec4-47ba-8884-3c609bd0e297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:54.384376 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a9941-eec4-47ba-8884-3c609bd0e297 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:54.392250 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20eb827-9a87-43e0-892c-b1afde575c3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:54.415103 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7889e-d30c-4dd9-8e7c-8d7a9d7f0610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:54.417520 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20eb827-9a87-43e0-892c-b1afde575c3f service cinder] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:54.419179 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7889e-d30c-4dd9-8e7c-8d7a9d7f0610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:54.426747 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7889e-d30c-4dd9-8e7c-8d7a9d7f0610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:54.429023 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7889e-d30c-4dd9-8e7c-8d7a9d7f0610 admin admin] PATH_INFO: `/v3/projects/18b55e0245c143dba1be592294e9f8af` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:54.433809 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc7889e-d30c-4dd9-8e7c-8d7a9d7f0610 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18b55e0245c143dba1be592294e9f8af)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:54.443578 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc7889e-d30c-4dd9-8e7c-8d7a9d7f0610 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:54.493400 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 863/1703] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:15:53 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 931 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:15:54.495948 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20eb827-9a87-43e0-892c-b1afde575c3f service cinder] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:54.540588 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20eb827-9a87-43e0-892c-b1afde575c3f service cinder] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:54.580373 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f047eb2-096e-4f31-8941-ac4956a9b650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:54.594545 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f047eb2-096e-4f31-8941-ac4956a9b650 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:54.596260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f047eb2-096e-4f31-8941-ac4956a9b650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:54.596260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f047eb2-096e-4f31-8941-ac4956a9b650 admin admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:54.596260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f047eb2-096e-4f31-8941-ac4956a9b650 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:15:54.604837 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f047eb2-096e-4f31-8941-ac4956a9b650 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:54.613738 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f047eb2-096e-4f31-8941-ac4956a9b650 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:54.617453 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-8bc7889e-d30c-4dd9-8e7c-8d7a9d7f0610 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18b55e0245c143dba1be592294e9f8af', 'request_id': 'req-8bc7889e-d30c-4dd9-8e7c-8d7a9d7f0610'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:54.652059 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-8bc7889e-d30c-4dd9-8e7c-8d7a9d7f0610 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18b55e0245c143dba1be592294e9f8af', 'request_id': 'req-8bc7889e-d30c-4dd9-8e7c-8d7a9d7f0610'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:54.687967 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-8bc7889e-d30c-4dd9-8e7c-8d7a9d7f0610 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18b55e0245c143dba1be592294e9f8af', 'request_id': 'req-8bc7889e-d30c-4dd9-8e7c-8d7a9d7f0610'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:54.692182 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 864/1704] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:15:54.693159 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f047eb2-096e-4f31-8941-ac4956a9b650 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:54.743670 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f047eb2-096e-4f31-8941-ac4956a9b650 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:54.748648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-8bc7889e-d30c-4dd9-8e7c-8d7a9d7f0610 admin admin] The token cache is being invalidate because project 18b55e0245c143dba1be592294e9f8af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:15:54.755688 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-8bc7889e-d30c-4dd9-8e7c-8d7a9d7f0610 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:54.757264 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-8bc7889e-d30c-4dd9-8e7c-8d7a9d7f0610 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:54.771087 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 841/1705] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:15:54 2022] DELETE /identity/v3/projects/18b55e0245c143dba1be592294e9f8af => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:15:54.772840 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cea863-3e3a-458a-8cdc-3152e6840b0a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:54.773171 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cea863-3e3a-458a-8cdc-3152e6840b0a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:54.773499 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cea863-3e3a-458a-8cdc-3152e6840b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:54.859247 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:54.888639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-18cea863-3e3a-458a-8cdc-3152e6840b0a None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:54.890108 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-18cea863-3e3a-458a-8cdc-3152e6840b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:54.983841 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a3fb3-f1ae-4586-a833-fbd1bf07a3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:54.991158 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a3fb3-f1ae-4586-a833-fbd1bf07a3e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:54.994086 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a3fb3-f1ae-4586-a833-fbd1bf07a3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:54.995544 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a3fb3-f1ae-4586-a833-fbd1bf07a3e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:54.997257 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a3fb3-f1ae-4586-a833-fbd1bf07a3e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:55.007872 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a3fb3-f1ae-4586-a833-fbd1bf07a3e2 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:55.114310 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 865/1706] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:15:54 2022] POST /identity/v3/projects => generated 387 bytes in 421 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:15:55.123062 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a3fb3-f1ae-4586-a833-fbd1bf07a3e2 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:55.136338 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-5b4a3fb3-f1ae-4586-a833-fbd1bf07a3e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:55.165361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 842/1707] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:55.177843 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff847e-56cb-42c6-b74d-157e9e8ee57e None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:55.178147 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff847e-56cb-42c6-b74d-157e9e8ee57e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:55.178453 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff847e-56cb-42c6-b74d-157e9e8ee57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:55.224107 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-16ff847e-56cb-42c6-b74d-157e9e8ee57e None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:55.225547 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-16ff847e-56cb-42c6-b74d-157e9e8ee57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:55.324016 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:55.367533 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 843/1708] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:55.369679 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff847e-56cb-42c6-b74d-157e9e8ee57e None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:55.412578 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff847e-56cb-42c6-b74d-157e9e8ee57e None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:55.444464 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a67ac-4e79-4dcc-8dca-2e39f13be064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:55.445432 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a67ac-4e79-4dcc-8dca-2e39f13be064 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:55.445787 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a67ac-4e79-4dcc-8dca-2e39f13be064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:55.446138 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a67ac-4e79-4dcc-8dca-2e39f13be064 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:55.447020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73a67ac-4e79-4dcc-8dca-2e39f13be064 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:15:55.447548 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73a67ac-4e79-4dcc-8dca-2e39f13be064 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:55.449851 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73a67ac-4e79-4dcc-8dca-2e39f13be064 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:55.506786 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 844/1709] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:15:55.509522 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a3fb3-f1ae-4586-a833-fbd1bf07a3e2 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:55.513839 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a67ac-4e79-4dcc-8dca-2e39f13be064 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:55.538639 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8dfe83-2c72-4496-8c17-5931fa3911f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:55.538639 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8dfe83-2c72-4496-8c17-5931fa3911f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:55.538639 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8dfe83-2c72-4496-8c17-5931fa3911f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:55.538639 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8dfe83-2c72-4496-8c17-5931fa3911f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:55.543280 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8dfe83-2c72-4496-8c17-5931fa3911f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:55.547066 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a67ac-4e79-4dcc-8dca-2e39f13be064 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:55.556991 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8dfe83-2c72-4496-8c17-5931fa3911f8 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:55.564377 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75eaff7-ba3f-4239-b6d5-c01d68553955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:55.565984 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75eaff7-ba3f-4239-b6d5-c01d68553955 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:55.566416 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75eaff7-ba3f-4239-b6d5-c01d68553955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:55.566859 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75eaff7-ba3f-4239-b6d5-c01d68553955 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:55.567916 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75eaff7-ba3f-4239-b6d5-c01d68553955 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:55.575274 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75eaff7-ba3f-4239-b6d5-c01d68553955 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:55.646478 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 866/1710] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:15:55 2022] POST /identity/v3/users => generated 294 bytes in 523 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:15:55.646700 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8dfe83-2c72-4496-8c17-5931fa3911f8 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:55.649863 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 845/1711] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:15:55 2022] POST /identity/v3/projects => generated 375 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:15:55.651139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75eaff7-ba3f-4239-b6d5-c01d68553955 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:55.683732 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8dfe83-2c72-4496-8c17-5931fa3911f8 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:55.846969 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574dc010-053b-4bb0-8c43-9e1c74793b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:55.849614 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574dc010-053b-4bb0-8c43-9e1c74793b90 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:55.850763 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574dc010-053b-4bb0-8c43-9e1c74793b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:55.851053 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574dc010-053b-4bb0-8c43-9e1c74793b90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:55.851866 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574dc010-053b-4bb0-8c43-9e1c74793b90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:55.859667 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574dc010-053b-4bb0-8c43-9e1c74793b90 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:55.900501 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 867/1712] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:15:55.901418 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574dc010-053b-4bb0-8c43-9e1c74793b90 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:55.911486 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-574dc010-053b-4bb0-8c43-9e1c74793b90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:55.955814 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574dc010-053b-4bb0-8c43-9e1c74793b90 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:55.986141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea04fe-2653-4ace-99f9-d96964090057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:55.987081 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea04fe-2653-4ace-99f9-d96964090057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:55.990035 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea04fe-2653-4ace-99f9-d96964090057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:55.990513 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea04fe-2653-4ace-99f9-d96964090057 admin admin] PATH_INFO: `/v3/users/ae72489e4aaa4253a2204eb1f66121e9` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:55.993721 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ea04fe-2653-4ace-99f9-d96964090057 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae72489e4aaa4253a2204eb1f66121e9)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:55.995755 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ea04fe-2653-4ace-99f9-d96964090057 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:56.056435 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75eaff7-ba3f-4239-b6d5-c01d68553955 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:56.083029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8db159c-d309-4c69-b40f-7ab5774bdd7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:56.093441 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db159c-d309-4c69-b40f-7ab5774bdd7b admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:56.101182 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db159c-d309-4c69-b40f-7ab5774bdd7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:56.102527 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db159c-d309-4c69-b40f-7ab5774bdd7b admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:56.103490 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8db159c-d309-4c69-b40f-7ab5774bdd7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:56.111727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8db159c-d309-4c69-b40f-7ab5774bdd7b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:56.189593 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:56.207460 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 846/1713] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:15:55 2022] POST /identity/v3/users => generated 288 bytes in 554 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:15:56.210962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8db159c-d309-4c69-b40f-7ab5774bdd7b admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:56.217787 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-56ea04fe-2653-4ace-99f9-d96964090057 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae72489e4aaa4253a2204eb1f66121e9'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:56.245333 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8db159c-d309-4c69-b40f-7ab5774bdd7b admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:56.258197 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-56ea04fe-2653-4ace-99f9-d96964090057 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae72489e4aaa4253a2204eb1f66121e9'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:56.285712 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc77672-88fe-4270-a984-e7602439deef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:56.287615 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc77672-88fe-4270-a984-e7602439deef admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:56.289240 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc77672-88fe-4270-a984-e7602439deef admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:56.289743 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc77672-88fe-4270-a984-e7602439deef admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:56.293568 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc77672-88fe-4270-a984-e7602439deef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:56.296676 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc77672-88fe-4270-a984-e7602439deef admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:56.325449 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-56ea04fe-2653-4ace-99f9-d96964090057 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae72489e4aaa4253a2204eb1f66121e9'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:56.360992 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-56ea04fe-2653-4ace-99f9-d96964090057 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae72489e4aaa4253a2204eb1f66121e9'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:56.363099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-56ea04fe-2653-4ace-99f9-d96964090057 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae72489e4aaa4253a2204eb1f66121e9'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:15:56.374638 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 847/1714] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:15:56.376881 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc77672-88fe-4270-a984-e7602439deef admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:56.384061 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7bc77672-88fe-4270-a984-e7602439deef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:56.404432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 868/1715] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:15:55 2022] DELETE /identity/v3/users/ae72489e4aaa4253a2204eb1f66121e9 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:15:56.405607 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea04fe-2653-4ace-99f9-d96964090057 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:56.410821 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-56ea04fe-2653-4ace-99f9-d96964090057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:56.428645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc77672-88fe-4270-a984-e7602439deef admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:56.642220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea04fe-2653-4ace-99f9-d96964090057 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:56.654621 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be891d-5619-4c67-a22d-c19d82d86782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:56.654908 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a772e9b-42f7-4786-845a-648d0693fb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:56.656020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be891d-5619-4c67-a22d-c19d82d86782 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:56.656150 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a772e9b-42f7-4786-845a-648d0693fb9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:56.656648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a772e9b-42f7-4786-845a-648d0693fb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:56.657107 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a772e9b-42f7-4786-845a-648d0693fb9f admin admin] PATH_INFO: `/v3/projects/6d671a472bd642ae897c73f7855c46fe/users/2538fd3c46034c62b8f4b62bdedfb1c7/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:56.657737 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be891d-5619-4c67-a22d-c19d82d86782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:56.660643 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be891d-5619-4c67-a22d-c19d82d86782 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:56.665347 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be891d-5619-4c67-a22d-c19d82d86782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:56.673888 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be891d-5619-4c67-a22d-c19d82d86782 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:56.713460 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a772e9b-42f7-4786-845a-648d0693fb9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d671a472bd642ae897c73f7855c46fe, user_id=2538fd3c46034c62b8f4b62bdedfb1c7, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:56.717324 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a772e9b-42f7-4786-845a-648d0693fb9f admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:56.719981 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 848/1716] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:15:56.729440 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0057098-7e08-4e7b-bfcd-a2386a3731d7 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:56.729996 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0057098-7e08-4e7b-bfcd-a2386a3731d7 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:56.737606 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0057098-7e08-4e7b-bfcd-a2386a3731d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:56.767829 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 869/1717] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:15:56 2022] PUT /identity/v3/projects/6d671a472bd642ae897c73f7855c46fe/users/2538fd3c46034c62b8f4b62bdedfb1c7/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:15:56.784465 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60b1d7-5b39-4af6-adf5-b7979d978f40 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:56.787084 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60b1d7-5b39-4af6-adf5-b7979d978f40 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:56.787084 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60b1d7-5b39-4af6-adf5-b7979d978f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:56.829167 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-e0057098-7e08-4e7b-bfcd-a2386a3731d7 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:56.833825 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e0057098-7e08-4e7b-bfcd-a2386a3731d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:56.914128 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:56.922228 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-2a60b1d7-5b39-4af6-adf5-b7979d978f40 None None] MFA Rules not processed for user `4206a21c317e4b5aace8541699f8032d`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:56.926255 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-2a60b1d7-5b39-4af6-adf5-b7979d978f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:57.079239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 849/1718] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:57.080220 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0057098-7e08-4e7b-bfcd-a2386a3731d7 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:57.107573 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0057098-7e08-4e7b-bfcd-a2386a3731d7 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:57.118460 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a586cd3c-f4dd-4c4e-9ece-d59708f77f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:57.119502 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a586cd3c-f4dd-4c4e-9ece-d59708f77f02 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:57.119876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a586cd3c-f4dd-4c4e-9ece-d59708f77f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:57.120157 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a586cd3c-f4dd-4c4e-9ece-d59708f77f02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:57.121997 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a586cd3c-f4dd-4c4e-9ece-d59708f77f02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:57.123733 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a586cd3c-f4dd-4c4e-9ece-d59708f77f02 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:57.165824 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 850/1719] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:15:57.167576 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f636208-e5ba-4056-b272-6dfd6446076a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:57.172097 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f636208-e5ba-4056-b272-6dfd6446076a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:57.172503 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f636208-e5ba-4056-b272-6dfd6446076a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:57.210452 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 870/1720] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 442 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:57.214078 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a60b1d7-5b39-4af6-adf5-b7979d978f40 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:57.229353 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-2a60b1d7-5b39-4af6-adf5-b7979d978f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:57.238837 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-0f636208-e5ba-4056-b272-6dfd6446076a None None] MFA Rules not processed for user `2538fd3c46034c62b8f4b62bdedfb1c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:57.240188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0f636208-e5ba-4056-b272-6dfd6446076a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:57.422274 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:57.430130 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 851/1721] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:57.437014 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f636208-e5ba-4056-b272-6dfd6446076a None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:57.467647 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f636208-e5ba-4056-b272-6dfd6446076a None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:57.475065 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c279b41b-42d7-4575-8ab7-cc51d492b74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:57.476028 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279b41b-42d7-4575-8ab7-cc51d492b74d admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:57.476694 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279b41b-42d7-4575-8ab7-cc51d492b74d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:57.478867 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279b41b-42d7-4575-8ab7-cc51d492b74d admin admin] PATH_INFO: `/v3/projects/6f708b764aed4b748bf934acc4e139eb/users/63a7efe3bf244e07b73a8d57acdc234e/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:57.498908 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c279b41b-42d7-4575-8ab7-cc51d492b74d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f708b764aed4b748bf934acc4e139eb, user_id=63a7efe3bf244e07b73a8d57acdc234e, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:57.502288 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c279b41b-42d7-4575-8ab7-cc51d492b74d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:57.553946 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 852/1722] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:15:57 2022] PUT /identity/v3/projects/6f708b764aed4b748bf934acc4e139eb/users/63a7efe3bf244e07b73a8d57acdc234e/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:15:57.556970 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887f79f-5549-4563-999f-ce7f079c50af None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:57.557448 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887f79f-5549-4563-999f-ce7f079c50af None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:57.558028 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887f79f-5549-4563-999f-ce7f079c50af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:57.564787 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a60b1d7-5b39-4af6-adf5-b7979d978f40 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:57.586443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-6887f79f-5549-4563-999f-ce7f079c50af None None] MFA Rules not processed for user `4206a21c317e4b5aace8541699f8032d`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:57.588695 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6887f79f-5549-4563-999f-ce7f079c50af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:57.735898 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 853/1723] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:57.741797 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887f79f-5549-4563-999f-ce7f079c50af None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:57.759432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3772539-f639-4085-bf13-0e1dce74ca4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:57.760106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3772539-f639-4085-bf13-0e1dce74ca4f service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:57.760401 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3772539-f639-4085-bf13-0e1dce74ca4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:57.760686 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3772539-f639-4085-bf13-0e1dce74ca4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:57.833635 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:57.865254 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3772539-f639-4085-bf13-0e1dce74ca4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:57.869552 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3772539-f639-4085-bf13-0e1dce74ca4f service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:57.885713 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 871/1724] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:15:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 675 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:15:57.888722 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902545e-762b-465f-9999-93486fb38614 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:57.889810 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902545e-762b-465f-9999-93486fb38614 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:57.890236 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902545e-762b-465f-9999-93486fb38614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:57.937254 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887f79f-5549-4563-999f-ce7f079c50af None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:57.942126 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-1902545e-762b-465f-9999-93486fb38614 None None] MFA Rules not processed for user `63a7efe3bf244e07b73a8d57acdc234e`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:57.942826 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225b004-29ae-4aa3-a6b5-ffa65d0791f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:57.943475 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225b004-29ae-4aa3-a6b5-ffa65d0791f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:57.943761 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225b004-29ae-4aa3-a6b5-ffa65d0791f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:57.943880 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-1902545e-762b-465f-9999-93486fb38614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:57.944032 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225b004-29ae-4aa3-a6b5-ffa65d0791f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:57.957926 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225b004-29ae-4aa3-a6b5-ffa65d0791f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:57.966852 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225b004-29ae-4aa3-a6b5-ffa65d0791f2 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:58.059569 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 854/1725] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:15:57 2022] POST /identity/v3/projects => generated 387 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:15:58.060700 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225b004-29ae-4aa3-a6b5-ffa65d0791f2 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:58.065743 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5225b004-29ae-4aa3-a6b5-ffa65d0791f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:58.187763 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 872/1726] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:58.188751 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1902545e-762b-465f-9999-93486fb38614 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:58.227979 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1902545e-762b-465f-9999-93486fb38614 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:58.297519 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:58.337245 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225b004-29ae-4aa3-a6b5-ffa65d0791f2 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:58.348877 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7910cd77-10fb-4498-9cc9-961a59b6dfa5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:58.349621 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7910cd77-10fb-4498-9cc9-961a59b6dfa5 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:58.349968 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7910cd77-10fb-4498-9cc9-961a59b6dfa5 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:58.350258 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7910cd77-10fb-4498-9cc9-961a59b6dfa5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:58.412080 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942cb05b-f9fb-4b62-ab2e-28ff481295a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:58.413161 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942cb05b-f9fb-4b62-ab2e-28ff481295a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:58.413886 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942cb05b-f9fb-4b62-ab2e-28ff481295a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:58.415609 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942cb05b-f9fb-4b62-ab2e-28ff481295a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:58.416452 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942cb05b-f9fb-4b62-ab2e-28ff481295a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:58.419711 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942cb05b-f9fb-4b62-ab2e-28ff481295a2 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:58.484539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 873/1727] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:15:58 2022] POST /identity/v3/users => generated 300 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:15:58.484539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942cb05b-f9fb-4b62-ab2e-28ff481295a2 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:58.528442 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942cb05b-f9fb-4b62-ab2e-28ff481295a2 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:58.546088 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7910cd77-10fb-4498-9cc9-961a59b6dfa5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:58.546565 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b997e-6016-4831-8705-a36dfcf78cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:58.548487 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b997e-6016-4831-8705-a36dfcf78cc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:58.548966 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b997e-6016-4831-8705-a36dfcf78cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:58.551076 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7910cd77-10fb-4498-9cc9-961a59b6dfa5 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:58.551311 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b997e-6016-4831-8705-a36dfcf78cc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:58.552562 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b997e-6016-4831-8705-a36dfcf78cc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:58.557251 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b997e-6016-4831-8705-a36dfcf78cc6 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:58.580962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 855/1728] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:15:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:15:58.594389 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a503b5-d0e7-41b0-b6ff-1afe675d7be9 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:58.599593 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a503b5-d0e7-41b0-b6ff-1afe675d7be9 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:58.601776 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a503b5-d0e7-41b0-b6ff-1afe675d7be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:58.663525 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 874/1729] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:15:58 2022] POST /identity/v3/projects => generated 375 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:15:58.666401 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b997e-6016-4831-8705-a36dfcf78cc6 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:58.671582 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-85a503b5-d0e7-41b0-b6ff-1afe675d7be9 None None] MFA Rules not processed for user `aeeaad0680074179b3d29abcf0d51515`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:58.674286 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-85a503b5-d0e7-41b0-b6ff-1afe675d7be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:58.788534 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:58.818052 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 856/1730] 10.222.0.118 () {58 vars in 1016 bytes} [Fri Jun 3 16:15:58 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:15:58.819099 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a503b5-d0e7-41b0-b6ff-1afe675d7be9 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:58.883752 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b997e-6016-4831-8705-a36dfcf78cc6 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:58.912452 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d4e414-5ff3-478f-a7f8-ddad26aee787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:58.913163 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4e414-5ff3-478f-a7f8-ddad26aee787 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:58.913464 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4e414-5ff3-478f-a7f8-ddad26aee787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:58.913751 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4e414-5ff3-478f-a7f8-ddad26aee787 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:58.916395 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d4e414-5ff3-478f-a7f8-ddad26aee787 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:58.922450 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d4e414-5ff3-478f-a7f8-ddad26aee787 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:58.952969 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 875/1731] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:15:58.956748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d4e414-5ff3-478f-a7f8-ddad26aee787 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:58.986666 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a503b5-d0e7-41b0-b6ff-1afe675d7be9 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:58.992916 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129bfa63-0c62-462c-b02c-e7a64c441212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:58.993862 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bfa63-0c62-462c-b02c-e7a64c441212 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:58.994665 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bfa63-0c62-462c-b02c-e7a64c441212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:58.994665 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bfa63-0c62-462c-b02c-e7a64c441212 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:58.995549 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129bfa63-0c62-462c-b02c-e7a64c441212 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:58.998240 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129bfa63-0c62-462c-b02c-e7a64c441212 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:58.999272 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d4e414-5ff3-478f-a7f8-ddad26aee787 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:59.022668 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00741406-e20a-4893-91fa-867b62a2d368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:59.024746 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00741406-e20a-4893-91fa-867b62a2d368 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:59.027952 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00741406-e20a-4893-91fa-867b62a2d368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:59.031373 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00741406-e20a-4893-91fa-867b62a2d368 admin admin] PATH_INFO: `/v3/projects/178bb27d64824bcf9ceb31761e21ff27/users/3c51ea94469147b69e6213b24f60704a/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:59.061685 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 857/1732] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:15:58 2022] POST /identity/v3/users => generated 294 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:15:59.066461 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129bfa63-0c62-462c-b02c-e7a64c441212 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:59.096898 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129bfa63-0c62-462c-b02c-e7a64c441212 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:59.107621 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00741406-e20a-4893-91fa-867b62a2d368 admin admin] RBAC: Authorizing `identity:create_grant(project_id=178bb27d64824bcf9ceb31761e21ff27, user_id=3c51ea94469147b69e6213b24f60704a, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:59.110484 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed5194-3ef6-4e20-a0a4-e34ab3e60275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:59.111460 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed5194-3ef6-4e20-a0a4-e34ab3e60275 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:59.111859 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed5194-3ef6-4e20-a0a4-e34ab3e60275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:59.112829 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed5194-3ef6-4e20-a0a4-e34ab3e60275 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:59.113694 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffed5194-3ef6-4e20-a0a4-e34ab3e60275 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:59.114405 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00741406-e20a-4893-91fa-867b62a2d368 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:59.118412 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffed5194-3ef6-4e20-a0a4-e34ab3e60275 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:59.159606 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 858/1733] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:15:59.163372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed5194-3ef6-4e20-a0a4-e34ab3e60275 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:59.169747 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 876/1734] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:15:58 2022] PUT /identity/v3/projects/178bb27d64824bcf9ceb31761e21ff27/users/3c51ea94469147b69e6213b24f60704a/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:15:59.180454 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00741406-e20a-4893-91fa-867b62a2d368 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:59.194270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed5194-3ef6-4e20-a0a4-e34ab3e60275 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:59.432379 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675eaf1c-4b94-429a-a7f5-3afc0dd842b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:59.433517 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675eaf1c-4b94-429a-a7f5-3afc0dd842b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:59.433851 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675eaf1c-4b94-429a-a7f5-3afc0dd842b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:59.433952 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675eaf1c-4b94-429a-a7f5-3afc0dd842b6 admin admin] PATH_INFO: `/v3/projects/9e2587f6bc2e4012b1e2ee6092b62041/users/08cc586421ec4002ba94bb753ade9234/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:59.445077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00741406-e20a-4893-91fa-867b62a2d368 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:59.458810 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736b3fe-e635-4fc3-a9f7-ce3c7115d56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:59.461866 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736b3fe-e635-4fc3-a9f7-ce3c7115d56e admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:59.462393 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736b3fe-e635-4fc3-a9f7-ce3c7115d56e admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:59.462677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736b3fe-e635-4fc3-a9f7-ce3c7115d56e admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:59.465942 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d736b3fe-e635-4fc3-a9f7-ce3c7115d56e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:59.470358 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675eaf1c-4b94-429a-a7f5-3afc0dd842b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e2587f6bc2e4012b1e2ee6092b62041, user_id=08cc586421ec4002ba94bb753ade9234, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:59.471222 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d736b3fe-e635-4fc3-a9f7-ce3c7115d56e admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:59.473123 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675eaf1c-4b94-429a-a7f5-3afc0dd842b6 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:59.491960 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:59.509173 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 877/1735] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:15:59.512703 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 859/1736] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:15:59 2022] PUT /identity/v3/projects/9e2587f6bc2e4012b1e2ee6092b62041/users/08cc586421ec4002ba94bb753ade9234/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:15:59.514870 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736b3fe-e635-4fc3-a9f7-ce3c7115d56e admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:59.517363 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675eaf1c-4b94-429a-a7f5-3afc0dd842b6 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:59.537675 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736b3fe-e635-4fc3-a9f7-ce3c7115d56e admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:59.653943 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5428a3a-41c8-4faf-a3dc-425cfdbdeb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:59.655031 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5428a3a-41c8-4faf-a3dc-425cfdbdeb7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:59.656073 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5428a3a-41c8-4faf-a3dc-425cfdbdeb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:59.656516 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5428a3a-41c8-4faf-a3dc-425cfdbdeb7c admin admin] PATH_INFO: `/v3/projects/178bb27d64824bcf9ceb31761e21ff27/users/3c51ea94469147b69e6213b24f60704a/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:59.659053 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5428a3a-41c8-4faf-a3dc-425cfdbdeb7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=178bb27d64824bcf9ceb31761e21ff27, user_id=3c51ea94469147b69e6213b24f60704a, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:59.665572 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5428a3a-41c8-4faf-a3dc-425cfdbdeb7c admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:59.686236 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 878/1737] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:15:59 2022] PUT /identity/v3/projects/178bb27d64824bcf9ceb31761e21ff27/users/3c51ea94469147b69e6213b24f60704a/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:15:59.700738 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5428a3a-41c8-4faf-a3dc-425cfdbdeb7c admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:59.723129 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:15:59.732803 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675eaf1c-4b94-429a-a7f5-3afc0dd842b6 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:59.844087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5428a3a-41c8-4faf-a3dc-425cfdbdeb7c admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:59.848784 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a2024a-057e-435b-b413-0900ed413e44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:59.849710 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2024a-057e-435b-b413-0900ed413e44 service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:59.850446 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2024a-057e-435b-b413-0900ed413e44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:59.851010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2024a-057e-435b-b413-0900ed413e44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:59.863006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a2024a-057e-435b-b413-0900ed413e44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:59.865237 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a2024a-057e-435b-b413-0900ed413e44 service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:59.877118 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff0d2f-758f-44b8-8ac2-ee8db9961b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:59.877882 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff0d2f-758f-44b8-8ac2-ee8db9961b6a admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:59.878171 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff0d2f-758f-44b8-8ac2-ee8db9961b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:59.879015 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff0d2f-758f-44b8-8ac2-ee8db9961b6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:59.880293 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ff0d2f-758f-44b8-8ac2-ee8db9961b6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:59.881554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ff0d2f-758f-44b8-8ac2-ee8db9961b6a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:59.887981 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 879/1738] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:15:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:15:59.893356 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13340e-a961-4e42-8aa0-fc7399dc8778 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:59.895501 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13340e-a961-4e42-8aa0-fc7399dc8778 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:59.896008 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13340e-a961-4e42-8aa0-fc7399dc8778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:59.904554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 860/1739] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:15:59.909142 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff0d2f-758f-44b8-8ac2-ee8db9961b6a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:15:59.927878 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff0d2f-758f-44b8-8ac2-ee8db9961b6a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:15:59.939130 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd19ab-69f4-4b13-82bb-7452f5043f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:15:59.941146 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd19ab-69f4-4b13-82bb-7452f5043f39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:59.941429 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd19ab-69f4-4b13-82bb-7452f5043f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:59.941776 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd19ab-69f4-4b13-82bb-7452f5043f39 admin admin] PATH_INFO: `/v3/projects/9e2587f6bc2e4012b1e2ee6092b62041/users/08cc586421ec4002ba94bb753ade9234/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:15:59.948324 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd19ab-69f4-4b13-82bb-7452f5043f39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e2587f6bc2e4012b1e2ee6092b62041, user_id=08cc586421ec4002ba94bb753ade9234, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:15:59.951231 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd19ab-69f4-4b13-82bb-7452f5043f39 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:15:59.976327 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-2d13340e-a961-4e42-8aa0-fc7399dc8778 None None] MFA Rules not processed for user `3c51ea94469147b69e6213b24f60704a`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:15:59.978822 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-2d13340e-a961-4e42-8aa0-fc7399dc8778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:15:59.988941 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 861/1740] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:15:59 2022] PUT /identity/v3/projects/9e2587f6bc2e4012b1e2ee6092b62041/users/08cc586421ec4002ba94bb753ade9234/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:15:59.991215 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e99e91-d2aa-4a70-8d6f-de6641d274fb None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:15:59.991447 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e99e91-d2aa-4a70-8d6f-de6641d274fb None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:15:59.991722 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e99e91-d2aa-4a70-8d6f-de6641d274fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:00.057373 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-a6e99e91-d2aa-4a70-8d6f-de6641d274fb None None] MFA Rules not processed for user `08cc586421ec4002ba94bb753ade9234`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:16:00.059794 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a6e99e91-d2aa-4a70-8d6f-de6641d274fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:00.236290 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 862/1741] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:59 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:16:00.236482 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 880/1742] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:15:59 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:16:00.237096 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e99e91-d2aa-4a70-8d6f-de6641d274fb None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:00.238644 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a6e99e91-d2aa-4a70-8d6f-de6641d274fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:00.243840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13340e-a961-4e42-8aa0-fc7399dc8778 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:00.251152 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-2d13340e-a961-4e42-8aa0-fc7399dc8778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:00.356043 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:16:00.375230 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e99e91-d2aa-4a70-8d6f-de6641d274fb None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:00.379660 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7519f20-516b-46ef-b1df-63dfca3e56f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:00.380330 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7519f20-516b-46ef-b1df-63dfca3e56f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:00.380612 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7519f20-516b-46ef-b1df-63dfca3e56f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:00.380904 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7519f20-516b-46ef-b1df-63dfca3e56f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:00.404750 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7519f20-516b-46ef-b1df-63dfca3e56f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:00.406223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7519f20-516b-46ef-b1df-63dfca3e56f9 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:00.428362 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 863/1743] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:16:00 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:16:00.506112 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7519f20-516b-46ef-b1df-63dfca3e56f9 service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:00.513602 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13340e-a961-4e42-8aa0-fc7399dc8778 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:00.530934 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53869e49-8d5f-4be8-ae3d-66a4f9decbe9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:00.535996 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53869e49-8d5f-4be8-ae3d-66a4f9decbe9 service cinder] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:00.536676 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:16:00.538584 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53869e49-8d5f-4be8-ae3d-66a4f9decbe9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:00.539733 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53869e49-8d5f-4be8-ae3d-66a4f9decbe9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:00.665404 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7519f20-516b-46ef-b1df-63dfca3e56f9 service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:00.684941 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710c87bf-8bd0-4030-a938-78e919191023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:00.686160 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c87bf-8bd0-4030-a938-78e919191023 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:00.686365 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c87bf-8bd0-4030-a938-78e919191023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:00.686692 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c87bf-8bd0-4030-a938-78e919191023 admin admin] PATH_INFO: `/v3/users/2538fd3c46034c62b8f4b62bdedfb1c7` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:00.689389 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710c87bf-8bd0-4030-a938-78e919191023 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2538fd3c46034c62b8f4b62bdedfb1c7)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:00.691385 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710c87bf-8bd0-4030-a938-78e919191023 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:00.741538 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53869e49-8d5f-4be8-ae3d-66a4f9decbe9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:00.743732 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53869e49-8d5f-4be8-ae3d-66a4f9decbe9 service cinder] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:00.756532 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 881/1744] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:16:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 16:16:00.824587 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-710c87bf-8bd0-4030-a938-78e919191023 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2538fd3c46034c62b8f4b62bdedfb1c7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:00.844389 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-710c87bf-8bd0-4030-a938-78e919191023 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2538fd3c46034c62b8f4b62bdedfb1c7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:00.888675 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-710c87bf-8bd0-4030-a938-78e919191023 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2538fd3c46034c62b8f4b62bdedfb1c7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:00.889599 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-710c87bf-8bd0-4030-a938-78e919191023 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2538fd3c46034c62b8f4b62bdedfb1c7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:00.901347 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-710c87bf-8bd0-4030-a938-78e919191023 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2538fd3c46034c62b8f4b62bdedfb1c7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:00.909069 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 864/1745] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:16:00 2022] DELETE /identity/v3/users/2538fd3c46034c62b8f4b62bdedfb1c7 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:16:00.913192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28620e-e183-4980-94dd-807b811b5186 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:00.913689 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28620e-e183-4980-94dd-807b811b5186 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:00.913801 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28620e-e183-4980-94dd-807b811b5186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:00.985476 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-9e28620e-e183-4980-94dd-807b811b5186 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:16:00.995969 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-9e28620e-e183-4980-94dd-807b811b5186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:01.203563 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8456c-9962-4f28-ba73-bae6e3b6b492 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:01.204382 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8456c-9962-4f28-ba73-bae6e3b6b492 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:01.204660 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8456c-9962-4f28-ba73-bae6e3b6b492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:01.255007 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 882/1746] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:16:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:16:01.262967 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e28620e-e183-4980-94dd-807b811b5186 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:01.265546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-9e28620e-e183-4980-94dd-807b811b5186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:01.305358 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-5ac8456c-9962-4f28-ba73-bae6e3b6b492 None None] MFA Rules not processed for user `c68189df1f9b4f9aacfa319381849697`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:16:01.306969 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e28620e-e183-4980-94dd-807b811b5186 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:01.307697 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5ac8456c-9962-4f28-ba73-bae6e3b6b492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:01.320580 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e90a2-54e4-43a6-8525-237dd55d72a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:01.322891 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e90a2-54e4-43a6-8525-237dd55d72a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:01.324085 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e90a2-54e4-43a6-8525-237dd55d72a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:01.324367 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e90a2-54e4-43a6-8525-237dd55d72a7 admin admin] PATH_INFO: `/v3/projects/3d3fc547ac324886a3cfb147650b4965` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:01.329291 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4e90a2-54e4-43a6-8525-237dd55d72a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d3fc547ac324886a3cfb147650b4965)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:01.333651 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4e90a2-54e4-43a6-8525-237dd55d72a7 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:01.469692 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-fe4e90a2-54e4-43a6-8525-237dd55d72a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d3fc547ac324886a3cfb147650b4965', 'request_id': 'req-fe4e90a2-54e4-43a6-8525-237dd55d72a7'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:01.497791 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-fe4e90a2-54e4-43a6-8525-237dd55d72a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d3fc547ac324886a3cfb147650b4965', 'request_id': 'req-fe4e90a2-54e4-43a6-8525-237dd55d72a7'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:01.515548 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-fe4e90a2-54e4-43a6-8525-237dd55d72a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d3fc547ac324886a3cfb147650b4965', 'request_id': 'req-fe4e90a2-54e4-43a6-8525-237dd55d72a7'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:01.563257 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 865/1747] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:16:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:16:01.563257 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac8456c-9962-4f28-ba73-bae6e3b6b492 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:01.569213 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5ac8456c-9962-4f28-ba73-bae6e3b6b492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:01.583150 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:16:01.597017 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-fe4e90a2-54e4-43a6-8525-237dd55d72a7 admin admin] The token cache is being invalidate because project 3d3fc547ac324886a3cfb147650b4965 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:16:01.597994 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-fe4e90a2-54e4-43a6-8525-237dd55d72a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:01.599008 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-fe4e90a2-54e4-43a6-8525-237dd55d72a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:01.609359 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 883/1748] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:16:01 2022] DELETE /identity/v3/projects/3d3fc547ac324886a3cfb147650b4965 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:16:01.615790 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e90a2-54e4-43a6-8525-237dd55d72a7 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:01.621956 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-fe4e90a2-54e4-43a6-8525-237dd55d72a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:01.631218 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:16:01.706230 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac8456c-9962-4f28-ba73-bae6e3b6b492 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:01.828034 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba42e5-b5a5-4101-ba41-64c797ca144b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:01.828817 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba42e5-b5a5-4101-ba41-64c797ca144b service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:01.829239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba42e5-b5a5-4101-ba41-64c797ca144b service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:01.829847 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba42e5-b5a5-4101-ba41-64c797ca144b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:01.833201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-63ba42e5-b5a5-4101-ba41-64c797ca144b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:02.077063 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ba42e5-b5a5-4101-ba41-64c797ca144b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:02.082655 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ba42e5-b5a5-4101-ba41-64c797ca144b service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:02.103844 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e90a2-54e4-43a6-8525-237dd55d72a7 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:02.124991 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143a612-2fa3-40a6-adc2-16a267ba46b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:02.128319 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143a612-2fa3-40a6-adc2-16a267ba46b0 service nova] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:02.128645 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143a612-2fa3-40a6-adc2-16a267ba46b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:02.128890 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143a612-2fa3-40a6-adc2-16a267ba46b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:02.129241 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 866/1749] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:16:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:16:02.130145 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba42e5-b5a5-4101-ba41-64c797ca144b service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:02.135148 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-63ba42e5-b5a5-4101-ba41-64c797ca144b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:02.137801 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-7143a612-2fa3-40a6-adc2-16a267ba46b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:02.178816 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba42e5-b5a5-4101-ba41-64c797ca144b service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:02.200084 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746319b-4148-444e-a201-fd82ba7f39e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:02.207817 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746319b-4148-444e-a201-fd82ba7f39e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:02.208417 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746319b-4148-444e-a201-fd82ba7f39e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:02.208909 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746319b-4148-444e-a201-fd82ba7f39e1 admin admin] PATH_INFO: `/v3/users/608c2aca5d514c758ca81f026d642b00` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:02.217813 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b746319b-4148-444e-a201-fd82ba7f39e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=608c2aca5d514c758ca81f026d642b00)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:02.222202 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b746319b-4148-444e-a201-fd82ba7f39e1 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:02.364603 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7143a612-2fa3-40a6-adc2-16a267ba46b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:02.366446 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7143a612-2fa3-40a6-adc2-16a267ba46b0 service nova] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:02.392260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 884/1750] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:16:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 782 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:16:02.396239 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930dd74-ffb8-48e7-a8e8-6d1dc150a47e None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:02.396693 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930dd74-ffb8-48e7-a8e8-6d1dc150a47e None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:02.397831 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930dd74-ffb8-48e7-a8e8-6d1dc150a47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:02.405367 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b746319b-4148-444e-a201-fd82ba7f39e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '608c2aca5d514c758ca81f026d642b00'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:02.442017 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b746319b-4148-444e-a201-fd82ba7f39e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '608c2aca5d514c758ca81f026d642b00'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:02.448535 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-8930dd74-ffb8-48e7-a8e8-6d1dc150a47e None None] MFA Rules not processed for user `94f91ae071b24e31a8ef8d50933ba4fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:16:02.450530 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-8930dd74-ffb8-48e7-a8e8-6d1dc150a47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:02.476048 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b746319b-4148-444e-a201-fd82ba7f39e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '608c2aca5d514c758ca81f026d642b00'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:02.476998 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b746319b-4148-444e-a201-fd82ba7f39e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '608c2aca5d514c758ca81f026d642b00'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:02.507522 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b746319b-4148-444e-a201-fd82ba7f39e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '608c2aca5d514c758ca81f026d642b00'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:02.513611 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:16:02.522742 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 867/1751] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:16:02 2022] DELETE /identity/v3/users/608c2aca5d514c758ca81f026d642b00 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:16:02.618434 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 885/1752] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:16:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:16:02.772663 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f7f1c-3c26-4043-b932-c3f0c8a58652 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:02.773227 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f7f1c-3c26-4043-b932-c3f0c8a58652 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:02.773713 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f7f1c-3c26-4043-b932-c3f0c8a58652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:02.811956 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-174f7f1c-3c26-4043-b932-c3f0c8a58652 None None] MFA Rules not processed for user `ef7c1e449b864b50b3c71c39f19968a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:16:02.813243 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-174f7f1c-3c26-4043-b932-c3f0c8a58652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:02.859861 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad994cfa-204a-4c8d-9384-0acb4403036e None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:02.860815 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad994cfa-204a-4c8d-9384-0acb4403036e None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:02.861333 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad994cfa-204a-4c8d-9384-0acb4403036e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:02.918079 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-ad994cfa-204a-4c8d-9384-0acb4403036e None None] MFA Rules not processed for user `4206a21c317e4b5aace8541699f8032d`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:16:02.923726 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-ad994cfa-204a-4c8d-9384-0acb4403036e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:03.101154 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:16:03.141735 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 868/1753] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:16:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:16:03.181637 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 886/1754] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:16:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:16:04.852847 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f7f1c-3c26-4043-b932-c3f0c8a58652 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:04.856309 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-174f7f1c-3c26-4043-b932-c3f0c8a58652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:05.001164 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f7f1c-3c26-4043-b932-c3f0c8a58652 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:05.025542 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053ddef-011d-4dcc-8c43-d72db4406441 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:05.026206 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f8675-7e65-4306-9adf-d6aad0cfde3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:05.026655 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053ddef-011d-4dcc-8c43-d72db4406441 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:05.027743 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053ddef-011d-4dcc-8c43-d72db4406441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:05.031325 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f8675-7e65-4306-9adf-d6aad0cfde3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:05.031623 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f8675-7e65-4306-9adf-d6aad0cfde3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:05.031915 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f8675-7e65-4306-9adf-d6aad0cfde3e admin admin] PATH_INFO: `/v3/projects/97daef81233647f48d7a1c4ff67cc313` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:05.038589 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8f8675-7e65-4306-9adf-d6aad0cfde3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=97daef81233647f48d7a1c4ff67cc313)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:05.046250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8f8675-7e65-4306-9adf-d6aad0cfde3e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:05.091170 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-3053ddef-011d-4dcc-8c43-d72db4406441 None None] MFA Rules not processed for user `aeeaad0680074179b3d29abcf0d51515`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:16:05.092694 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-3053ddef-011d-4dcc-8c43-d72db4406441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:05.258489 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0a8f8675-7e65-4306-9adf-d6aad0cfde3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97daef81233647f48d7a1c4ff67cc313', 'request_id': 'req-0a8f8675-7e65-4306-9adf-d6aad0cfde3e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:05.272065 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 887/1755] 10.222.0.41 () {58 vars in 1009 bytes} [Fri Jun 3 16:16:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:16:05.273783 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85214d4b-b299-4baa-9ff5-a8d600c3e61e None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:05.274084 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85214d4b-b299-4baa-9ff5-a8d600c3e61e None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:05.274370 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85214d4b-b299-4baa-9ff5-a8d600c3e61e None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:05.275694 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 888/1756] 10.222.0.41 () {54 vars in 921 bytes} [Fri Jun 3 16:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:16:05.278629 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0a8f8675-7e65-4306-9adf-d6aad0cfde3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97daef81233647f48d7a1c4ff67cc313', 'request_id': 'req-0a8f8675-7e65-4306-9adf-d6aad0cfde3e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:05.291409 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85214d4b-b299-4baa-9ff5-a8d600c3e61e None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:05.294307 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-85214d4b-b299-4baa-9ff5-a8d600c3e61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:05.304699 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0a8f8675-7e65-4306-9adf-d6aad0cfde3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97daef81233647f48d7a1c4ff67cc313', 'request_id': 'req-0a8f8675-7e65-4306-9adf-d6aad0cfde3e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:05.331172 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0a8f8675-7e65-4306-9adf-d6aad0cfde3e admin admin] The token cache is being invalidate because project 97daef81233647f48d7a1c4ff67cc313 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:16:05.331685 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0a8f8675-7e65-4306-9adf-d6aad0cfde3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:05.332423 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0a8f8675-7e65-4306-9adf-d6aad0cfde3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:05.341714 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 869/1757] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:16:04 2022] DELETE /identity/v3/projects/97daef81233647f48d7a1c4ff67cc313 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:16:05.346365 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c393ca25-f67e-486f-acac-9e2160a176ed None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:05.346660 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c393ca25-f67e-486f-acac-9e2160a176ed None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:05.346953 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c393ca25-f67e-486f-acac-9e2160a176ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:05.412534 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c393ca25-f67e-486f-acac-9e2160a176ed None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:16:05.418558 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c393ca25-f67e-486f-acac-9e2160a176ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:05.450602 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85214d4b-b299-4baa-9ff5-a8d600c3e61e None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:05.454707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b14297-cac2-4642-8b70-a3434372ee64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:05.455486 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b14297-cac2-4642-8b70-a3434372ee64 service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:05.455779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b14297-cac2-4642-8b70-a3434372ee64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:05.456064 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b14297-cac2-4642-8b70-a3434372ee64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:05.458007 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-09b14297-cac2-4642-8b70-a3434372ee64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:05.471962 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b14297-cac2-4642-8b70-a3434372ee64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:05.473786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b14297-cac2-4642-8b70-a3434372ee64 service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:05.492264 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 889/1758] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:16:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:16:05.493993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b14297-cac2-4642-8b70-a3434372ee64 service neutron] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:05.495492 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-09b14297-cac2-4642-8b70-a3434372ee64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:05.526191 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:16:05.688125 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 870/1759] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:16:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:16:05.699464 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c65616-bb6d-4e41-915a-3ff88b7342c2 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:05.700838 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c65616-bb6d-4e41-915a-3ff88b7342c2 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:05.703381 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c65616-bb6d-4e41-915a-3ff88b7342c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:05.721473 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b14297-cac2-4642-8b70-a3434372ee64 service neutron] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:05.744117 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca3d09-ab1f-487b-80fd-92cb04c0167c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:05.745151 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca3d09-ab1f-487b-80fd-92cb04c0167c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:05.746060 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca3d09-ab1f-487b-80fd-92cb04c0167c admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:05.746632 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca3d09-ab1f-487b-80fd-92cb04c0167c admin admin] PATH_INFO: `/v3/projects/6d671a472bd642ae897c73f7855c46fe` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:05.747900 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca3d09-ab1f-487b-80fd-92cb04c0167c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d671a472bd642ae897c73f7855c46fe)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:05.749974 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca3d09-ab1f-487b-80fd-92cb04c0167c admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:05.751227 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-11c65616-bb6d-4e41-915a-3ff88b7342c2 None None] MFA Rules not processed for user `2fd08ea39ee94b92ac6763608e69f158`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:16:05.752654 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-11c65616-bb6d-4e41-915a-3ff88b7342c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:05.913846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:16:05.938050 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-e6ca3d09-ab1f-487b-80fd-92cb04c0167c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d671a472bd642ae897c73f7855c46fe', 'request_id': 'req-e6ca3d09-ab1f-487b-80fd-92cb04c0167c'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:05.964218 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-e6ca3d09-ab1f-487b-80fd-92cb04c0167c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d671a472bd642ae897c73f7855c46fe', 'request_id': 'req-e6ca3d09-ab1f-487b-80fd-92cb04c0167c'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:05.982556 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 871/1760] 10.222.0.41 () {58 vars in 1018 bytes} [Fri Jun 3 16:16:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:16:05.987412 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06adf62c-de5b-431c-8377-06ee9980889a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:05.988722 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06adf62c-de5b-431c-8377-06ee9980889a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:05.990352 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06adf62c-de5b-431c-8377-06ee9980889a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:06.000628 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-e6ca3d09-ab1f-487b-80fd-92cb04c0167c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d671a472bd642ae897c73f7855c46fe', 'request_id': 'req-e6ca3d09-ab1f-487b-80fd-92cb04c0167c'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:06.026971 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-e6ca3d09-ab1f-487b-80fd-92cb04c0167c admin admin] The token cache is being invalidate because project 6d671a472bd642ae897c73f7855c46fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:16:06.026971 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-e6ca3d09-ab1f-487b-80fd-92cb04c0167c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:06.033396 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-e6ca3d09-ab1f-487b-80fd-92cb04c0167c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:06.044179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 890/1761] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:16:05 2022] DELETE /identity/v3/projects/6d671a472bd642ae897c73f7855c46fe => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:16:06.045462 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca3d09-ab1f-487b-80fd-92cb04c0167c admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:06.052520 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-e6ca3d09-ab1f-487b-80fd-92cb04c0167c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:06.057494 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-06adf62c-de5b-431c-8377-06ee9980889a None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:16:06.062528 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-06adf62c-de5b-431c-8377-06ee9980889a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:06.299232 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 872/1762] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:16:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:16:06.305350 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06adf62c-de5b-431c-8377-06ee9980889a None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:06.329230 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06adf62c-de5b-431c-8377-06ee9980889a None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:06.329961 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca3d09-ab1f-487b-80fd-92cb04c0167c admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:06.341739 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03884619-806e-4044-bade-41c60928d8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:06.342433 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03884619-806e-4044-bade-41c60928d8e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:06.342718 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03884619-806e-4044-bade-41c60928d8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:06.344077 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03884619-806e-4044-bade-41c60928d8e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:06.344841 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03884619-806e-4044-bade-41c60928d8e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:16:06.345159 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03884619-806e-4044-bade-41c60928d8e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:06.349676 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03884619-806e-4044-bade-41c60928d8e7 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:06.350119 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2fb11f-42db-45f0-8245-e766fdb3b556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:06.351043 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2fb11f-42db-45f0-8245-e766fdb3b556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:06.352578 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2fb11f-42db-45f0-8245-e766fdb3b556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:06.352895 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2fb11f-42db-45f0-8245-e766fdb3b556 admin admin] PATH_INFO: `/v3/users/3c51ea94469147b69e6213b24f60704a` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:06.360611 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2fb11f-42db-45f0-8245-e766fdb3b556 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c51ea94469147b69e6213b24f60704a)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:06.362589 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2fb11f-42db-45f0-8245-e766fdb3b556 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:06.387672 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 873/1763] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:16:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:16:06.388678 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03884619-806e-4044-bade-41c60928d8e7 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:06.428728 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03884619-806e-4044-bade-41c60928d8e7 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:06.442151 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef11feef-d1cc-4c9c-9b04-40e829092e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:06.444124 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11feef-d1cc-4c9c-9b04-40e829092e72 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:06.446107 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11feef-d1cc-4c9c-9b04-40e829092e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:06.446286 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11feef-d1cc-4c9c-9b04-40e829092e72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:06.451525 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef11feef-d1cc-4c9c-9b04-40e829092e72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:06.456823 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef11feef-d1cc-4c9c-9b04-40e829092e72 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:06.502135 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-1b2fb11f-42db-45f0-8245-e766fdb3b556 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c51ea94469147b69e6213b24f60704a'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:06.507124 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 874/1764] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:16:06 2022] POST /identity/v3/projects => generated 401 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:16:06.511273 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef11feef-d1cc-4c9c-9b04-40e829092e72 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:06.522955 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-1b2fb11f-42db-45f0-8245-e766fdb3b556 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c51ea94469147b69e6213b24f60704a'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:06.540772 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-1b2fb11f-42db-45f0-8245-e766fdb3b556 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c51ea94469147b69e6213b24f60704a'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:06.550629 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-1b2fb11f-42db-45f0-8245-e766fdb3b556 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c51ea94469147b69e6213b24f60704a'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:06.551956 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-1b2fb11f-42db-45f0-8245-e766fdb3b556 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c51ea94469147b69e6213b24f60704a'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:06.567422 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 891/1765] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:16:06 2022] DELETE /identity/v3/users/3c51ea94469147b69e6213b24f60704a => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:16:06.622635 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:16:06.657274 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef11feef-d1cc-4c9c-9b04-40e829092e72 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:06.738674 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fdef41-39d5-4ac4-a9f2-37994effc3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:06.739472 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fdef41-39d5-4ac4-a9f2-37994effc3df admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:06.740928 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fdef41-39d5-4ac4-a9f2-37994effc3df admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:06.741342 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fdef41-39d5-4ac4-a9f2-37994effc3df admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:06.742183 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fdef41-39d5-4ac4-a9f2-37994effc3df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:06.745246 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fdef41-39d5-4ac4-a9f2-37994effc3df admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:06.776770 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 875/1766] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:16:06 2022] POST /identity/v3/users => generated 301 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:16:06.780573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2fb11f-42db-45f0-8245-e766fdb3b556 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:06.782384 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-1b2fb11f-42db-45f0-8245-e766fdb3b556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:06.807677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2fb11f-42db-45f0-8245-e766fdb3b556 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:06.826795 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1b37d-8f3c-4c33-81e0-de65ed2f6640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:06.827578 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1b37d-8f3c-4c33-81e0-de65ed2f6640 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:06.827808 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1b37d-8f3c-4c33-81e0-de65ed2f6640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:06.828091 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1b37d-8f3c-4c33-81e0-de65ed2f6640 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:06.828826 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1b37d-8f3c-4c33-81e0-de65ed2f6640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:06.832009 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1b37d-8f3c-4c33-81e0-de65ed2f6640 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:06.860391 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fdef41-39d5-4ac4-a9f2-37994effc3df admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:06.872041 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 892/1767] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:16:06.873966 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fdef41-39d5-4ac4-a9f2-37994effc3df admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:06.894967 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaebd089-4660-409e-a318-19abc35f43d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:06.896099 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebd089-4660-409e-a318-19abc35f43d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:06.897967 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebd089-4660-409e-a318-19abc35f43d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:06.902266 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebd089-4660-409e-a318-19abc35f43d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:06.903766 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaebd089-4660-409e-a318-19abc35f43d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:06.905679 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaebd089-4660-409e-a318-19abc35f43d3 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:06.935480 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 876/1768] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:16:06.938637 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1b37d-8f3c-4c33-81e0-de65ed2f6640 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:06.967821 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1b37d-8f3c-4c33-81e0-de65ed2f6640 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:06.974540 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4fb88a-35e2-42ca-b5c2-95d50fd4c155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:06.975719 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4fb88a-35e2-42ca-b5c2-95d50fd4c155 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:06.976115 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4fb88a-35e2-42ca-b5c2-95d50fd4c155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:06.976498 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4fb88a-35e2-42ca-b5c2-95d50fd4c155 admin admin] PATH_INFO: `/v3/projects/80a8543538824ac0aed7caa24ae6685d/users/fafb7e001a56484ba9f6da3152955919/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:07.022096 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4fb88a-35e2-42ca-b5c2-95d50fd4c155 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80a8543538824ac0aed7caa24ae6685d, user_id=fafb7e001a56484ba9f6da3152955919, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:07.026117 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4fb88a-35e2-42ca-b5c2-95d50fd4c155 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:07.055334 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 893/1769] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:16:06 2022] PUT /identity/v3/projects/80a8543538824ac0aed7caa24ae6685d/users/fafb7e001a56484ba9f6da3152955919/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:16:07.064965 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8760f9-5bc9-4e07-89a2-5a730c9a1760 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:07.069283 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8760f9-5bc9-4e07-89a2-5a730c9a1760 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:07.069767 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8760f9-5bc9-4e07-89a2-5a730c9a1760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:07.140253 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-1e8760f9-5bc9-4e07-89a2-5a730c9a1760 None None] MFA Rules not processed for user `fafb7e001a56484ba9f6da3152955919`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:16:07.145325 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1e8760f9-5bc9-4e07-89a2-5a730c9a1760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:07.294709 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 877/1770] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:16:07 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:16:07.300545 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4fb88a-35e2-42ca-b5c2-95d50fd4c155 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:07.326407 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:16:07.454031 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4fb88a-35e2-42ca-b5c2-95d50fd4c155 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:07.461769 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b1013-0b86-4bf8-bb58-1cfe9c9263bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:07.463195 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b1013-0b86-4bf8-bb58-1cfe9c9263bd admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:07.463773 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b1013-0b86-4bf8-bb58-1cfe9c9263bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:07.464280 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b1013-0b86-4bf8-bb58-1cfe9c9263bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:07.466863 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5b1013-0b86-4bf8-bb58-1cfe9c9263bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:07.486518 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5b1013-0b86-4bf8-bb58-1cfe9c9263bd admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:07.494125 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8760f9-5bc9-4e07-89a2-5a730c9a1760 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:07.496049 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1e8760f9-5bc9-4e07-89a2-5a730c9a1760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:07.523937 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8760f9-5bc9-4e07-89a2-5a730c9a1760 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:07.533702 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e6e657-c852-4732-a683-7a4f2c6573e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:07.535902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6e657-c852-4732-a683-7a4f2c6573e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:07.536747 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6e657-c852-4732-a683-7a4f2c6573e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:07.536747 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6e657-c852-4732-a683-7a4f2c6573e6 admin admin] PATH_INFO: `/v3/projects/178bb27d64824bcf9ceb31761e21ff27` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:07.538480 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e6e657-c852-4732-a683-7a4f2c6573e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=178bb27d64824bcf9ceb31761e21ff27)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:07.541322 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e6e657-c852-4732-a683-7a4f2c6573e6 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:07.560035 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 894/1771] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:16:07 2022] POST /identity/v3/projects => generated 399 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:16:07.562520 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b1013-0b86-4bf8-bb58-1cfe9c9263bd admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:07.568097 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-fb5b1013-0b86-4bf8-bb58-1cfe9c9263bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:07.629228 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:16:07.635952 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-31e6e657-c852-4732-a683-7a4f2c6573e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '178bb27d64824bcf9ceb31761e21ff27', 'request_id': 'req-31e6e657-c852-4732-a683-7a4f2c6573e6'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:07.646694 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-31e6e657-c852-4732-a683-7a4f2c6573e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '178bb27d64824bcf9ceb31761e21ff27', 'request_id': 'req-31e6e657-c852-4732-a683-7a4f2c6573e6'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:07.672106 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-31e6e657-c852-4732-a683-7a4f2c6573e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '178bb27d64824bcf9ceb31761e21ff27', 'request_id': 'req-31e6e657-c852-4732-a683-7a4f2c6573e6'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:07.691023 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-31e6e657-c852-4732-a683-7a4f2c6573e6 admin admin] The token cache is being invalidate because project 178bb27d64824bcf9ceb31761e21ff27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:16:07.691503 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-31e6e657-c852-4732-a683-7a4f2c6573e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:07.694563 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-31e6e657-c852-4732-a683-7a4f2c6573e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:07.696954 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 878/1772] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:16:07 2022] DELETE /identity/v3/projects/178bb27d64824bcf9ceb31761e21ff27 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:16:07.697893 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e6e657-c852-4732-a683-7a4f2c6573e6 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:07.701627 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-31e6e657-c852-4732-a683-7a4f2c6573e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:07.754216 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b1013-0b86-4bf8-bb58-1cfe9c9263bd admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:07.814786 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e6e657-c852-4732-a683-7a4f2c6573e6 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:07.820729 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980dc4bd-d692-4521-b427-96fbb386463b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:07.822189 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980dc4bd-d692-4521-b427-96fbb386463b admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:07.822652 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980dc4bd-d692-4521-b427-96fbb386463b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:07.823126 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980dc4bd-d692-4521-b427-96fbb386463b admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:07.824897 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980dc4bd-d692-4521-b427-96fbb386463b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:07.827352 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980dc4bd-d692-4521-b427-96fbb386463b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:07.865692 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 879/1773] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:16:07 2022] POST /identity/v3/users => generated 306 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:16:07.868682 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566f070-4e51-4f71-9084-9e398799a0a6 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:07.868978 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566f070-4e51-4f71-9084-9e398799a0a6 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:07.869877 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566f070-4e51-4f71-9084-9e398799a0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:07.898851 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb8a6a-af8f-43d2-8aa7-15bbb1c56342 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:07.899068 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb8a6a-af8f-43d2-8aa7-15bbb1c56342 service nova] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:07.899950 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb8a6a-af8f-43d2-8aa7-15bbb1c56342 service nova] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:07.899950 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb8a6a-af8f-43d2-8aa7-15bbb1c56342 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:07.903652 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-9dbb8a6a-af8f-43d2-8aa7-15bbb1c56342 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:07.912136 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-4566f070-4e51-4f71-9084-9e398799a0a6 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:16:07.916343 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4566f070-4e51-4f71-9084-9e398799a0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:07.930162 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb8a6a-af8f-43d2-8aa7-15bbb1c56342 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:07.931794 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb8a6a-af8f-43d2-8aa7-15bbb1c56342 service nova] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:07.940946 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 880/1774] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:16:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:16:07.940946 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4566f070-4e51-4f71-9084-9e398799a0a6 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:07.956856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4566f070-4e51-4f71-9084-9e398799a0a6 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:07.960093 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 895/1775] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:16:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:16:07.976381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551dbcc-5b5e-40a3-8bb3-1c13a38c5cec None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:07.976618 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551dbcc-5b5e-40a3-8bb3-1c13a38c5cec None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:07.976884 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551dbcc-5b5e-40a3-8bb3-1c13a38c5cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:07.985218 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9c9b1-cd43-4d99-92bb-4e106afc8241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:07.986405 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9c9b1-cd43-4d99-92bb-4e106afc8241 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:07.986694 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9c9b1-cd43-4d99-92bb-4e106afc8241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:07.987000 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9c9b1-cd43-4d99-92bb-4e106afc8241 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:07.987719 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a9c9b1-cd43-4d99-92bb-4e106afc8241 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:07.995962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a9c9b1-cd43-4d99-92bb-4e106afc8241 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:08.023695 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 881/1776] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:16:07 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:16:08.025954 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9c9b1-cd43-4d99-92bb-4e106afc8241 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:08.030511 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-1551dbcc-5b5e-40a3-8bb3-1c13a38c5cec None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:16:08.031777 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-1551dbcc-5b5e-40a3-8bb3-1c13a38c5cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:08.041903 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9c9b1-cd43-4d99-92bb-4e106afc8241 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:08.051488 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 896/1777] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:16:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:16:08.052782 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1551dbcc-5b5e-40a3-8bb3-1c13a38c5cec None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:08.056156 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f91769-b287-48cb-8c88-c58ff4ae1bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:08.057070 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f91769-b287-48cb-8c88-c58ff4ae1bc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:08.060169 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f91769-b287-48cb-8c88-c58ff4ae1bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:08.060462 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f91769-b287-48cb-8c88-c58ff4ae1bc8 admin admin] PATH_INFO: `/v3/projects/869c6fa14446408fbf45126064a532a6/users/c6cffecabdc24a2890b79838fa91e82c/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:08.081135 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1551dbcc-5b5e-40a3-8bb3-1c13a38c5cec None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:08.091631 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018a689-47f3-4141-8020-fa4221587d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:08.093085 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018a689-47f3-4141-8020-fa4221587d2a admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:08.093567 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018a689-47f3-4141-8020-fa4221587d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:08.094031 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018a689-47f3-4141-8020-fa4221587d2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:08.096684 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1018a689-47f3-4141-8020-fa4221587d2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:16:08.097777 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1018a689-47f3-4141-8020-fa4221587d2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:08.100168 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1018a689-47f3-4141-8020-fa4221587d2a admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:08.101694 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f91769-b287-48cb-8c88-c58ff4ae1bc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=869c6fa14446408fbf45126064a532a6, user_id=c6cffecabdc24a2890b79838fa91e82c, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:08.112902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f91769-b287-48cb-8c88-c58ff4ae1bc8 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:08.150415 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 897/1778] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:16:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:16:08.154082 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018a689-47f3-4141-8020-fa4221587d2a admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:08.158206 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 882/1779] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:16:08 2022] PUT /identity/v3/projects/869c6fa14446408fbf45126064a532a6/users/c6cffecabdc24a2890b79838fa91e82c/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:16:08.163973 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f91769-b287-48cb-8c88-c58ff4ae1bc8 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:08.171109 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:16:08.187498 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018a689-47f3-4141-8020-fa4221587d2a admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:08.331951 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f91769-b287-48cb-8c88-c58ff4ae1bc8 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:08.333297 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426509a-6bdc-407f-94b7-726485bc9c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:08.333980 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426509a-6bdc-407f-94b7-726485bc9c1f admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:08.334258 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426509a-6bdc-407f-94b7-726485bc9c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:08.334711 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426509a-6bdc-407f-94b7-726485bc9c1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:08.335586 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a426509a-6bdc-407f-94b7-726485bc9c1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:08.338566 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a426509a-6bdc-407f-94b7-726485bc9c1f admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:08.348394 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f33a3f-5966-4433-aa88-43d76a59a2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:08.349396 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f33a3f-5966-4433-aa88-43d76a59a2a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:08.349855 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f33a3f-5966-4433-aa88-43d76a59a2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:08.350301 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f33a3f-5966-4433-aa88-43d76a59a2a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:08.351218 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f33a3f-5966-4433-aa88-43d76a59a2a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:08.355265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f33a3f-5966-4433-aa88-43d76a59a2a3 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:08.380742 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 883/1780] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:16:08.384518 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f33a3f-5966-4433-aa88-43d76a59a2a3 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:08.423540 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 898/1781] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:16:08 2022] POST /identity/v3/projects => generated 369 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:16:08.426686 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f33a3f-5966-4433-aa88-43d76a59a2a3 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:08.427668 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426509a-6bdc-407f-94b7-726485bc9c1f admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:08.468479 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:16:08.617998 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426509a-6bdc-407f-94b7-726485bc9c1f admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:08.623185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704c7afd-fd32-4ea9-98da-a645c3f5b1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:08.623185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704c7afd-fd32-4ea9-98da-a645c3f5b1c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:08.623185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704c7afd-fd32-4ea9-98da-a645c3f5b1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:08.623594 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704c7afd-fd32-4ea9-98da-a645c3f5b1c3 admin admin] PATH_INFO: `/v3/projects/869c6fa14446408fbf45126064a532a6/users/c6cffecabdc24a2890b79838fa91e82c/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:08.625503 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704c7afd-fd32-4ea9-98da-a645c3f5b1c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=869c6fa14446408fbf45126064a532a6, user_id=c6cffecabdc24a2890b79838fa91e82c, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:08.628855 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704c7afd-fd32-4ea9-98da-a645c3f5b1c3 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:08.642128 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000013ea-2d17-4083-bceb-f5a5ede758dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:08.643976 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000013ea-2d17-4083-bceb-f5a5ede758dc admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:08.644312 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000013ea-2d17-4083-bceb-f5a5ede758dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:08.644602 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000013ea-2d17-4083-bceb-f5a5ede758dc admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:08.647810 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000013ea-2d17-4083-bceb-f5a5ede758dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:08.649800 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000013ea-2d17-4083-bceb-f5a5ede758dc admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:08.662899 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 884/1782] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:16:08 2022] PUT /identity/v3/projects/869c6fa14446408fbf45126064a532a6/users/c6cffecabdc24a2890b79838fa91e82c/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:16:08.674191 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7e6b2-a7e4-4a36-95a6-342f9dcd7a57 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:08.674398 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7e6b2-a7e4-4a36-95a6-342f9dcd7a57 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:08.674706 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7e6b2-a7e4-4a36-95a6-342f9dcd7a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:08.725992 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 899/1783] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:16:08 2022] POST /identity/v3/users => generated 285 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:16:08.727197 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000013ea-2d17-4083-bceb-f5a5ede758dc admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:08.760230 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-9bd7e6b2-a7e4-4a36-95a6-342f9dcd7a57 None None] MFA Rules not processed for user `c6cffecabdc24a2890b79838fa91e82c`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:16:08.762619 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9bd7e6b2-a7e4-4a36-95a6-342f9dcd7a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:08.763996 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000013ea-2d17-4083-bceb-f5a5ede758dc admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:08.928021 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28d47c-9534-4f00-8bb5-0b18b20f339e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:08.929174 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28d47c-9534-4f00-8bb5-0b18b20f339e admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:08.929605 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28d47c-9534-4f00-8bb5-0b18b20f339e admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:08.930053 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28d47c-9534-4f00-8bb5-0b18b20f339e admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:08.931010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff28d47c-9534-4f00-8bb5-0b18b20f339e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:08.932746 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff28d47c-9534-4f00-8bb5-0b18b20f339e admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:08.940532 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 885/1784] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:16:08 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:16:08.951981 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd7e6b2-a7e4-4a36-95a6-342f9dcd7a57 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:08.953618 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9bd7e6b2-a7e4-4a36-95a6-342f9dcd7a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:08.966060 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 900/1785] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:16:08.969876 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28d47c-9534-4f00-8bb5-0b18b20f339e admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:08.991248 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28d47c-9534-4f00-8bb5-0b18b20f339e admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:09.004491 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7aa6b-4ba8-46b9-945e-0d003e8dc11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:09.005168 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7aa6b-4ba8-46b9-945e-0d003e8dc11b admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:09.005457 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7aa6b-4ba8-46b9-945e-0d003e8dc11b admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:09.005739 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7aa6b-4ba8-46b9-945e-0d003e8dc11b admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:09.006619 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf7aa6b-4ba8-46b9-945e-0d003e8dc11b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:09.008292 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf7aa6b-4ba8-46b9-945e-0d003e8dc11b admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:09.032542 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 901/1786] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:16:09.033969 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca6e4b-16f2-49cd-932a-dab2b810fa2c None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:09.034267 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca6e4b-16f2-49cd-932a-dab2b810fa2c None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:09.034551 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca6e4b-16f2-49cd-932a-dab2b810fa2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:09.041810 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:16:09.053244 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd7e6b2-a7e4-4a36-95a6-342f9dcd7a57 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:09.058872 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29694a14-61f6-49b4-830b-b7ef3a94b90a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:09.059755 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29694a14-61f6-49b4-830b-b7ef3a94b90a service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:09.060049 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29694a14-61f6-49b4-830b-b7ef3a94b90a service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:09.060329 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29694a14-61f6-49b4-830b-b7ef3a94b90a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:09.068565 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-f0ca6e4b-16f2-49cd-932a-dab2b810fa2c None None] MFA Rules not processed for user `c68189df1f9b4f9aacfa319381849697`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:16:09.071067 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-f0ca6e4b-16f2-49cd-932a-dab2b810fa2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:09.081524 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29694a14-61f6-49b4-830b-b7ef3a94b90a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:09.084785 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29694a14-61f6-49b4-830b-b7ef3a94b90a service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:09.103986 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 886/1787] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:16:08 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:16:09.106951 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29694a14-61f6-49b4-830b-b7ef3a94b90a service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:09.124894 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29694a14-61f6-49b4-830b-b7ef3a94b90a service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:09.138866 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de412b60-25c8-487e-b51e-ca32ac82aed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:09.140209 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de412b60-25c8-487e-b51e-ca32ac82aed7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:09.144803 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de412b60-25c8-487e-b51e-ca32ac82aed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:09.147161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de412b60-25c8-487e-b51e-ca32ac82aed7 admin admin] PATH_INFO: `/v3/projects/f006fe57505d43bbb0cdfd8d739f385c/users/6afcba283b764ee69f8e3335908f6d71/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:09.166931 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 902/1788] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:16:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:16:09.192662 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de412b60-25c8-487e-b51e-ca32ac82aed7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f006fe57505d43bbb0cdfd8d739f385c, user_id=6afcba283b764ee69f8e3335908f6d71, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:09.197044 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de412b60-25c8-487e-b51e-ca32ac82aed7 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:09.227714 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 887/1789] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:16:09 2022] PUT /identity/v3/projects/f006fe57505d43bbb0cdfd8d739f385c/users/6afcba283b764ee69f8e3335908f6d71/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:16:09.244156 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a022bc9f-7892-4a71-a43d-b539ab407448 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:09.251781 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a022bc9f-7892-4a71-a43d-b539ab407448 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:09.252191 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a022bc9f-7892-4a71-a43d-b539ab407448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:09.263367 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de412b60-25c8-487e-b51e-ca32ac82aed7 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:09.297178 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-a022bc9f-7892-4a71-a43d-b539ab407448 None None] MFA Rules not processed for user `6afcba283b764ee69f8e3335908f6d71`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:16:09.298002 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:16:09.300890 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-a022bc9f-7892-4a71-a43d-b539ab407448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:09.381640 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de412b60-25c8-487e-b51e-ca32ac82aed7 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:09.389039 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b50b0c-8597-4722-a7fe-e0e394c11204 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:09.390454 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b50b0c-8597-4722-a7fe-e0e394c11204 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:09.391280 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b50b0c-8597-4722-a7fe-e0e394c11204 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:09.391792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b50b0c-8597-4722-a7fe-e0e394c11204 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:09.394328 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-41b50b0c-8597-4722-a7fe-e0e394c11204 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:09.395083 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 903/1790] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:16:09 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:16:09.407841 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a022bc9f-7892-4a71-a43d-b539ab407448 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:09.409905 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-a022bc9f-7892-4a71-a43d-b539ab407448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:09.491186 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b50b0c-8597-4722-a7fe-e0e394c11204 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:09.492688 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b50b0c-8597-4722-a7fe-e0e394c11204 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:09.505218 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 888/1791] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:16:09 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:16:09.549022 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a022bc9f-7892-4a71-a43d-b539ab407448 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:09.554571 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb497095-7c25-4220-b138-085df82db863 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbaf66d6e1f54047917cc615546bbd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:09.555245 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb497095-7c25-4220-b138-085df82db863 service glance] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:09.555561 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb497095-7c25-4220-b138-085df82db863 service glance] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:09.555864 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb497095-7c25-4220-b138-085df82db863 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:09.567647 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb497095-7c25-4220-b138-085df82db863 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:09.569086 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb497095-7c25-4220-b138-085df82db863 service glance] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:09.583131 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 904/1792] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:16:09 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:16:14.439745 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b50b0c-8597-4722-a7fe-e0e394c11204 service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:14.449153 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b50b0c-8597-4722-a7fe-e0e394c11204 service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:14.456472 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc118d1-f83f-4694-8293-b62640b51122 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:14.465004 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc118d1-f83f-4694-8293-b62640b51122 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:14.465722 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc118d1-f83f-4694-8293-b62640b51122 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:14.466427 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc118d1-f83f-4694-8293-b62640b51122 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:14.471107 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9fc118d1-f83f-4694-8293-b62640b51122 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:14.549306 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc118d1-f83f-4694-8293-b62640b51122 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:14.550792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc118d1-f83f-4694-8293-b62640b51122 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:14.560329 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 889/1793] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:16:14 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:16:19.577129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=82 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:16:22.317827 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb497095-7c25-4220-b138-085df82db863 service glance] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:22.377772 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb497095-7c25-4220-b138-085df82db863 service glance] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:22.383012 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c860c5-485e-41c9-8bdb-10334f2e9a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:22.384036 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c860c5-485e-41c9-8bdb-10334f2e9a14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:22.384528 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c860c5-485e-41c9-8bdb-10334f2e9a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:22.384880 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c860c5-485e-41c9-8bdb-10334f2e9a14 admin admin] PATH_INFO: `/v3/users/fafb7e001a56484ba9f6da3152955919` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:22.386444 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c860c5-485e-41c9-8bdb-10334f2e9a14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fafb7e001a56484ba9f6da3152955919)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:22.388744 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c860c5-485e-41c9-8bdb-10334f2e9a14 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:22.438625 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-e1c860c5-485e-41c9-8bdb-10334f2e9a14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fafb7e001a56484ba9f6da3152955919'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:22.445888 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-e1c860c5-485e-41c9-8bdb-10334f2e9a14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fafb7e001a56484ba9f6da3152955919'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:22.457470 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-e1c860c5-485e-41c9-8bdb-10334f2e9a14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fafb7e001a56484ba9f6da3152955919'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:22.464153 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-e1c860c5-485e-41c9-8bdb-10334f2e9a14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fafb7e001a56484ba9f6da3152955919'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:22.464780 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-e1c860c5-485e-41c9-8bdb-10334f2e9a14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fafb7e001a56484ba9f6da3152955919'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:22.470634 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 905/1794] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:16:22 2022] DELETE /identity/v3/users/fafb7e001a56484ba9f6da3152955919 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:16:22.475245 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799dca81-d983-4140-a9f8-be2fc1fe2fa0 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:22.475728 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799dca81-d983-4140-a9f8-be2fc1fe2fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:22.475815 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799dca81-d983-4140-a9f8-be2fc1fe2fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:22.495799 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-799dca81-d983-4140-a9f8-be2fc1fe2fa0 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:16:22.497909 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-799dca81-d983-4140-a9f8-be2fc1fe2fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:22.560429 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 890/1795] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:16:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:16:22.570253 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c860c5-485e-41c9-8bdb-10334f2e9a14 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:22.571974 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-e1c860c5-485e-41c9-8bdb-10334f2e9a14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:22.637371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c860c5-485e-41c9-8bdb-10334f2e9a14 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:22.641776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13beb88c-1a78-4e7e-a588-871154c3793e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:22.642605 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13beb88c-1a78-4e7e-a588-871154c3793e service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:22.642892 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13beb88c-1a78-4e7e-a588-871154c3793e service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:22.643170 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13beb88c-1a78-4e7e-a588-871154c3793e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:22.652556 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13beb88c-1a78-4e7e-a588-871154c3793e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:22.654920 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13beb88c-1a78-4e7e-a588-871154c3793e service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:22.665141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 906/1796] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:16:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:16:22.994376 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799dca81-d983-4140-a9f8-be2fc1fe2fa0 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:22.996165 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-799dca81-d983-4140-a9f8-be2fc1fe2fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:23.003863 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:16:23.010798 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799dca81-d983-4140-a9f8-be2fc1fe2fa0 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:23.016149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864895f-f066-453e-883d-54618d230f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:23.017014 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864895f-f066-453e-883d-54618d230f95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:23.017298 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864895f-f066-453e-883d-54618d230f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:23.017575 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864895f-f066-453e-883d-54618d230f95 admin admin] PATH_INFO: `/v3/projects/80a8543538824ac0aed7caa24ae6685d` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:23.018757 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864895f-f066-453e-883d-54618d230f95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80a8543538824ac0aed7caa24ae6685d)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:23.020743 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864895f-f066-453e-883d-54618d230f95 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:23.074062 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a864895f-f066-453e-883d-54618d230f95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80a8543538824ac0aed7caa24ae6685d', 'request_id': 'req-a864895f-f066-453e-883d-54618d230f95'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:23.080202 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a864895f-f066-453e-883d-54618d230f95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80a8543538824ac0aed7caa24ae6685d', 'request_id': 'req-a864895f-f066-453e-883d-54618d230f95'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:23.087151 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a864895f-f066-453e-883d-54618d230f95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80a8543538824ac0aed7caa24ae6685d', 'request_id': 'req-a864895f-f066-453e-883d-54618d230f95'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:23.097246 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a864895f-f066-453e-883d-54618d230f95 admin admin] The token cache is being invalidate because project 80a8543538824ac0aed7caa24ae6685d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:16:23.097536 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a864895f-f066-453e-883d-54618d230f95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:23.098168 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a864895f-f066-453e-883d-54618d230f95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:23.099754 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 891/1797] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:16:22 2022] DELETE /identity/v3/projects/80a8543538824ac0aed7caa24ae6685d => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:16:23.103747 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13beb88c-1a78-4e7e-a588-871154c3793e service neutron] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:23.115648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13beb88c-1a78-4e7e-a588-871154c3793e service neutron] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:23.135582 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:16:23.189718 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869ec42-5b4c-46fe-8fb3-363c10114251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:23.190737 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869ec42-5b4c-46fe-8fb3-363c10114251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:23.191062 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869ec42-5b4c-46fe-8fb3-363c10114251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:23.191453 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869ec42-5b4c-46fe-8fb3-363c10114251 admin admin] PATH_INFO: `/v3/users/c6cffecabdc24a2890b79838fa91e82c` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:23.192811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1869ec42-5b4c-46fe-8fb3-363c10114251 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6cffecabdc24a2890b79838fa91e82c)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:23.194705 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1869ec42-5b4c-46fe-8fb3-363c10114251 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:23.244263 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-1869ec42-5b4c-46fe-8fb3-363c10114251 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6cffecabdc24a2890b79838fa91e82c'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:23.256663 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-1869ec42-5b4c-46fe-8fb3-363c10114251 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6cffecabdc24a2890b79838fa91e82c'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:23.279592 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-1869ec42-5b4c-46fe-8fb3-363c10114251 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6cffecabdc24a2890b79838fa91e82c'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:23.288011 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-1869ec42-5b4c-46fe-8fb3-363c10114251 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6cffecabdc24a2890b79838fa91e82c'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:23.288867 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-1869ec42-5b4c-46fe-8fb3-363c10114251 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6cffecabdc24a2890b79838fa91e82c'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:23.300541 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 907/1798] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:16:23 2022] DELETE /identity/v3/users/c6cffecabdc24a2890b79838fa91e82c => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:16:23.619885 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864895f-f066-453e-883d-54618d230f95 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:23.621406 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a864895f-f066-453e-883d-54618d230f95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:23.640147 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864895f-f066-453e-883d-54618d230f95 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:23.697314 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22fcfb-bf22-4f12-8b52-435f6f5b4583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:23.698201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22fcfb-bf22-4f12-8b52-435f6f5b4583 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:23.698486 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22fcfb-bf22-4f12-8b52-435f6f5b4583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:23.698763 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22fcfb-bf22-4f12-8b52-435f6f5b4583 admin admin] PATH_INFO: `/v3/projects/869c6fa14446408fbf45126064a532a6` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:23.699990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a22fcfb-bf22-4f12-8b52-435f6f5b4583 admin admin] RBAC: Authorizing `identity:delete_project(project_id=869c6fa14446408fbf45126064a532a6)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:23.701903 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a22fcfb-bf22-4f12-8b52-435f6f5b4583 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:23.752298 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7a22fcfb-bf22-4f12-8b52-435f6f5b4583 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '869c6fa14446408fbf45126064a532a6', 'request_id': 'req-7a22fcfb-bf22-4f12-8b52-435f6f5b4583'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:23.758609 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7a22fcfb-bf22-4f12-8b52-435f6f5b4583 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '869c6fa14446408fbf45126064a532a6', 'request_id': 'req-7a22fcfb-bf22-4f12-8b52-435f6f5b4583'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:23.767188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7a22fcfb-bf22-4f12-8b52-435f6f5b4583 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '869c6fa14446408fbf45126064a532a6', 'request_id': 'req-7a22fcfb-bf22-4f12-8b52-435f6f5b4583'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:23.777447 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7a22fcfb-bf22-4f12-8b52-435f6f5b4583 admin admin] The token cache is being invalidate because project 869c6fa14446408fbf45126064a532a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:16:23.777736 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7a22fcfb-bf22-4f12-8b52-435f6f5b4583 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:23.779111 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7a22fcfb-bf22-4f12-8b52-435f6f5b4583 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:23.780700 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 892/1799] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:16:23 2022] DELETE /identity/v3/projects/869c6fa14446408fbf45126064a532a6 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:16:23.786088 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e6cd0-f4a6-4d56-ae2f-d2878b1cab53 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:23.786584 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e6cd0-f4a6-4d56-ae2f-d2878b1cab53 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:23.787108 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e6cd0-f4a6-4d56-ae2f-d2878b1cab53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:23.808837 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-475e6cd0-f4a6-4d56-ae2f-d2878b1cab53 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:16:23.810055 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-475e6cd0-f4a6-4d56-ae2f-d2878b1cab53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:23.898505 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 908/1800] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:16:23.908166 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9894d-ca24-4d01-9fb1-c1721d443f58 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:23.908465 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9894d-ca24-4d01-9fb1-c1721d443f58 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:23.908748 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9894d-ca24-4d01-9fb1-c1721d443f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:23.948080 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-35f9894d-ca24-4d01-9fb1-c1721d443f58 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:16:23.949846 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-35f9894d-ca24-4d01-9fb1-c1721d443f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:23.963447 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 893/1801] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:16:23.968409 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e6cd0-f4a6-4d56-ae2f-d2878b1cab53 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:23.984710 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e6cd0-f4a6-4d56-ae2f-d2878b1cab53 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:23.991144 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f86cbd-dec9-4584-95e8-e1c426208062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:23.991921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f86cbd-dec9-4584-95e8-e1c426208062 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:23.992734 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f86cbd-dec9-4584-95e8-e1c426208062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:23.992734 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f86cbd-dec9-4584-95e8-e1c426208062 admin admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:23.993220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f86cbd-dec9-4584-95e8-e1c426208062 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:16:23.993533 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f86cbd-dec9-4584-95e8-e1c426208062 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:23.995980 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f86cbd-dec9-4584-95e8-e1c426208062 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:24.014414 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 909/1802] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:16:24.018921 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9894d-ca24-4d01-9fb1-c1721d443f58 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:24.029249 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9894d-ca24-4d01-9fb1-c1721d443f58 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:24.033896 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eacbf4-5f65-414b-a423-f0f6a6d10131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:24.034552 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eacbf4-5f65-414b-a423-f0f6a6d10131 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:24.034838 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eacbf4-5f65-414b-a423-f0f6a6d10131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:24.035114 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eacbf4-5f65-414b-a423-f0f6a6d10131 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:24.035847 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eacbf4-5f65-414b-a423-f0f6a6d10131 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:24.037748 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eacbf4-5f65-414b-a423-f0f6a6d10131 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:24.042283 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:16:24.071092 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 894/1803] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:16:24 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:16:24.075848 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f86cbd-dec9-4584-95e8-e1c426208062 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:24.085598 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f86cbd-dec9-4584-95e8-e1c426208062 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:24.111634 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:16:24.144956 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f133ca6-25a5-4146-9f83-357da0c2678f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:24.145624 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f133ca6-25a5-4146-9f83-357da0c2678f admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:24.145905 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f133ca6-25a5-4146-9f83-357da0c2678f admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:24.146209 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f133ca6-25a5-4146-9f83-357da0c2678f admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:24.146915 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f133ca6-25a5-4146-9f83-357da0c2678f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:24.148850 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f133ca6-25a5-4146-9f83-357da0c2678f admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:24.171990 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 910/1804] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:16:24 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:16:24.173097 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eacbf4-5f65-414b-a423-f0f6a6d10131 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:24.183103 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eacbf4-5f65-414b-a423-f0f6a6d10131 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:24.188279 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798ccd0d-653e-4502-9eed-0b4cb4eeabee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:24.188934 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ccd0d-653e-4502-9eed-0b4cb4eeabee admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:24.189221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ccd0d-653e-4502-9eed-0b4cb4eeabee admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:24.189499 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ccd0d-653e-4502-9eed-0b4cb4eeabee admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:24.190233 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798ccd0d-653e-4502-9eed-0b4cb4eeabee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:24.191860 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798ccd0d-653e-4502-9eed-0b4cb4eeabee admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:24.204926 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 895/1805] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:16:24.211063 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f133ca6-25a5-4146-9f83-357da0c2678f admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:24.221469 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f133ca6-25a5-4146-9f83-357da0c2678f admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:24.226289 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95398e31-2023-407d-a53b-407c2202b424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:24.226986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95398e31-2023-407d-a53b-407c2202b424 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:24.227282 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95398e31-2023-407d-a53b-407c2202b424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:24.227583 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95398e31-2023-407d-a53b-407c2202b424 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:24.228477 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95398e31-2023-407d-a53b-407c2202b424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:24.230189 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95398e31-2023-407d-a53b-407c2202b424 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:24.240934 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 911/1806] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:16:24.244915 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798ccd0d-653e-4502-9eed-0b4cb4eeabee admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:24.255767 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798ccd0d-653e-4502-9eed-0b4cb4eeabee admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:24.260247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e1c97-7949-4b36-800c-dff3543b2ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:24.261121 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e1c97-7949-4b36-800c-dff3543b2ba3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:24.261406 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e1c97-7949-4b36-800c-dff3543b2ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:24.261683 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e1c97-7949-4b36-800c-dff3543b2ba3 admin admin] PATH_INFO: `/v3/projects/425d20cf1c3c440f8bbaf2273f1e5fd8/users/32ec8edca1484528ad4fd578c055d471/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:24.276357 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e1c97-7949-4b36-800c-dff3543b2ba3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=425d20cf1c3c440f8bbaf2273f1e5fd8, user_id=32ec8edca1484528ad4fd578c055d471, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:24.279327 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e1c97-7949-4b36-800c-dff3543b2ba3 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:24.291876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 896/1807] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:16:24 2022] PUT /identity/v3/projects/425d20cf1c3c440f8bbaf2273f1e5fd8/users/32ec8edca1484528ad4fd578c055d471/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:16:24.302123 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66da7b2-977f-4b32-b557-f072e10538c5 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:24.302644 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66da7b2-977f-4b32-b557-f072e10538c5 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:24.302963 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66da7b2-977f-4b32-b557-f072e10538c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:24.330160 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-d66da7b2-977f-4b32-b557-f072e10538c5 None None] MFA Rules not processed for user `32ec8edca1484528ad4fd578c055d471`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:16:24.331415 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-d66da7b2-977f-4b32-b557-f072e10538c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:24.415855 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 912/1808] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:16:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:16:24.426557 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e1c97-7949-4b36-800c-dff3543b2ba3 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:24.428357 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4b8e1c97-7949-4b36-800c-dff3543b2ba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:24.496076 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e1c97-7949-4b36-800c-dff3543b2ba3 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:24.500287 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea0cb8-9686-4c00-910c-f82effc1b545 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:24.500935 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea0cb8-9686-4c00-910c-f82effc1b545 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:24.501219 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea0cb8-9686-4c00-910c-f82effc1b545 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:24.501496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea0cb8-9686-4c00-910c-f82effc1b545 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:24.509646 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea0cb8-9686-4c00-910c-f82effc1b545 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:24.511065 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea0cb8-9686-4c00-910c-f82effc1b545 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:24.520371 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 897/1809] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:16:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:16:31.799397 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc55d1-fc37-41bf-9f16-8e5d3c936e24 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:31.800096 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc55d1-fc37-41bf-9f16-8e5d3c936e24 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:31.800385 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc55d1-fc37-41bf-9f16-8e5d3c936e24 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:31.802138 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 913/1810] 10.222.0.41 () {54 vars in 921 bytes} [Fri Jun 3 16:16:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:16:31.811456 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece1921-c66c-444c-878e-e95467f6ac0f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:31.812000 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece1921-c66c-444c-878e-e95467f6ac0f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:31.812000 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece1921-c66c-444c-878e-e95467f6ac0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:31.857851 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-cece1921-c66c-444c-878e-e95467f6ac0f None None] MFA Rules not processed for user `2fd08ea39ee94b92ac6763608e69f158`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:16:31.859095 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-cece1921-c66c-444c-878e-e95467f6ac0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:31.996274 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 898/1811] 10.222.0.41 () {58 vars in 1018 bytes} [Fri Jun 3 16:16:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:16:32.084403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc55d1-fc37-41bf-9f16-8e5d3c936e24 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:32.087905 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-9acc55d1-fc37-41bf-9f16-8e5d3c936e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:32.114539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc55d1-fc37-41bf-9f16-8e5d3c936e24 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:32.122049 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66936727-c0fe-4943-8916-d25b10dbaa3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:32.123064 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66936727-c0fe-4943-8916-d25b10dbaa3b service nova] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:32.123490 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66936727-c0fe-4943-8916-d25b10dbaa3b service nova] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:32.123937 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66936727-c0fe-4943-8916-d25b10dbaa3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:32.165197 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66936727-c0fe-4943-8916-d25b10dbaa3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:32.167041 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66936727-c0fe-4943-8916-d25b10dbaa3b service nova] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:32.185756 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 914/1812] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:16:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:16:33.047913 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5547d-748e-42ad-bd6f-0edd8156dbeb None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:33.048410 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5547d-748e-42ad-bd6f-0edd8156dbeb None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:33.048484 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5547d-748e-42ad-bd6f-0edd8156dbeb None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:33.049903 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 899/1813] 10.222.0.41 () {54 vars in 921 bytes} [Fri Jun 3 16:16:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:16:33.055768 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6166e11-bb16-4a58-84c6-af9e651be547 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:33.056252 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6166e11-bb16-4a58-84c6-af9e651be547 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:33.056498 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6166e11-bb16-4a58-84c6-af9e651be547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:33.098822 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-e6166e11-bb16-4a58-84c6-af9e651be547 None None] MFA Rules not processed for user `2fd08ea39ee94b92ac6763608e69f158`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:16:33.100078 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-e6166e11-bb16-4a58-84c6-af9e651be547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:33.110083 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 915/1814] 10.222.0.41 () {58 vars in 1018 bytes} [Fri Jun 3 16:16:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:16:33.121972 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea5547d-748e-42ad-bd6f-0edd8156dbeb None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:33.136796 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea5547d-748e-42ad-bd6f-0edd8156dbeb None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:33.142876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98260faa-a098-4d1f-b3f3-df5f2d4cd75b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:33.143777 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98260faa-a098-4d1f-b3f3-df5f2d4cd75b service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:33.144208 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98260faa-a098-4d1f-b3f3-df5f2d4cd75b service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:33.144629 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98260faa-a098-4d1f-b3f3-df5f2d4cd75b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:33.155910 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98260faa-a098-4d1f-b3f3-df5f2d4cd75b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:33.157783 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98260faa-a098-4d1f-b3f3-df5f2d4cd75b service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:33.169880 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 900/1815] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:16:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:16:43.070942 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=62,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:16:43.165117 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=83,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:16:46.108956 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6166e11-bb16-4a58-84c6-af9e651be547 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:46.184408 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6166e11-bb16-4a58-84c6-af9e651be547 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:46.189817 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca37c8d1-b5da-404e-a990-40af2080cd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:46.190778 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37c8d1-b5da-404e-a990-40af2080cd2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:46.191133 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37c8d1-b5da-404e-a990-40af2080cd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:46.191510 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37c8d1-b5da-404e-a990-40af2080cd2c admin admin] PATH_INFO: `/v3/users/32ec8edca1484528ad4fd578c055d471` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:46.202123 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca37c8d1-b5da-404e-a990-40af2080cd2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=32ec8edca1484528ad4fd578c055d471)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:46.202123 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca37c8d1-b5da-404e-a990-40af2080cd2c admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:46.265594 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-ca37c8d1-b5da-404e-a990-40af2080cd2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32ec8edca1484528ad4fd578c055d471'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:46.284194 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-ca37c8d1-b5da-404e-a990-40af2080cd2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32ec8edca1484528ad4fd578c055d471'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:46.301136 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-ca37c8d1-b5da-404e-a990-40af2080cd2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32ec8edca1484528ad4fd578c055d471'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:46.315717 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-ca37c8d1-b5da-404e-a990-40af2080cd2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32ec8edca1484528ad4fd578c055d471'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:46.316469 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-ca37c8d1-b5da-404e-a990-40af2080cd2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32ec8edca1484528ad4fd578c055d471'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:46.324016 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 916/1816] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:16:46 2022] DELETE /identity/v3/users/32ec8edca1484528ad4fd578c055d471 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:16:46.328169 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca774b-af28-4062-a87c-e7848e8e7f41 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:46.328887 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca774b-af28-4062-a87c-e7848e8e7f41 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:46.328887 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca774b-af28-4062-a87c-e7848e8e7f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:46.350037 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-3eca774b-af28-4062-a87c-e7848e8e7f41 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:16:46.351288 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3eca774b-af28-4062-a87c-e7848e8e7f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:46.436001 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 901/1817] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:16:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:16:46.457777 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca37c8d1-b5da-404e-a990-40af2080cd2c admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:46.457777 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-ca37c8d1-b5da-404e-a990-40af2080cd2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:46.527175 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca37c8d1-b5da-404e-a990-40af2080cd2c admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:46.531387 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7766ae2-60c7-4598-a7d6-f99a0dfbe113 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:46.532089 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7766ae2-60c7-4598-a7d6-f99a0dfbe113 service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:46.532379 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7766ae2-60c7-4598-a7d6-f99a0dfbe113 service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:46.532659 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7766ae2-60c7-4598-a7d6-f99a0dfbe113 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:46.541731 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7766ae2-60c7-4598-a7d6-f99a0dfbe113 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:46.543160 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7766ae2-60c7-4598-a7d6-f99a0dfbe113 service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:46.553418 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 917/1818] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:16:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:16:46.898430 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca774b-af28-4062-a87c-e7848e8e7f41 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:46.899929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3eca774b-af28-4062-a87c-e7848e8e7f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:46.915823 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca774b-af28-4062-a87c-e7848e8e7f41 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:46.921775 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aae17b-e73e-4321-978a-fae22e43eff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:46.922640 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aae17b-e73e-4321-978a-fae22e43eff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:46.922927 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aae17b-e73e-4321-978a-fae22e43eff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:46.923207 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aae17b-e73e-4321-978a-fae22e43eff3 admin admin] PATH_INFO: `/v3/projects/425d20cf1c3c440f8bbaf2273f1e5fd8` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:46.924860 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aae17b-e73e-4321-978a-fae22e43eff3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=425d20cf1c3c440f8bbaf2273f1e5fd8)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:46.926855 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aae17b-e73e-4321-978a-fae22e43eff3 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:46.972798 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-13aae17b-e73e-4321-978a-fae22e43eff3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '425d20cf1c3c440f8bbaf2273f1e5fd8', 'request_id': 'req-13aae17b-e73e-4321-978a-fae22e43eff3'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:46.978729 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-13aae17b-e73e-4321-978a-fae22e43eff3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '425d20cf1c3c440f8bbaf2273f1e5fd8', 'request_id': 'req-13aae17b-e73e-4321-978a-fae22e43eff3'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:46.986672 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-13aae17b-e73e-4321-978a-fae22e43eff3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '425d20cf1c3c440f8bbaf2273f1e5fd8', 'request_id': 'req-13aae17b-e73e-4321-978a-fae22e43eff3'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:46.996669 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-13aae17b-e73e-4321-978a-fae22e43eff3 admin admin] The token cache is being invalidate because project 425d20cf1c3c440f8bbaf2273f1e5fd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:16:46.996952 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-13aae17b-e73e-4321-978a-fae22e43eff3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:46.997568 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-13aae17b-e73e-4321-978a-fae22e43eff3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:46.999090 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 902/1819] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:16:46 2022] DELETE /identity/v3/projects/425d20cf1c3c440f8bbaf2273f1e5fd8 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:16:47.004121 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584063e9-f2af-4967-96a3-77ee95ed1415 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:47.004416 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584063e9-f2af-4967-96a3-77ee95ed1415 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:47.004700 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584063e9-f2af-4967-96a3-77ee95ed1415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:47.022443 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-584063e9-f2af-4967-96a3-77ee95ed1415 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:16:47.023800 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-584063e9-f2af-4967-96a3-77ee95ed1415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:47.041414 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:16:47.085484 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 918/1820] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:16:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:16:47.093709 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1597c5-82e3-4086-8233-c2826d5bda3b None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:47.094002 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1597c5-82e3-4086-8233-c2826d5bda3b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:47.094287 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1597c5-82e3-4086-8233-c2826d5bda3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:47.113086 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-eb1597c5-82e3-4086-8233-c2826d5bda3b None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:16:47.114321 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-eb1597c5-82e3-4086-8233-c2826d5bda3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:47.122047 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 903/1821] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:16:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:16:47.126146 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584063e9-f2af-4967-96a3-77ee95ed1415 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:47.135783 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584063e9-f2af-4967-96a3-77ee95ed1415 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:47.140775 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026cc810-1743-4f82-a334-9abc9484850a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:47.141422 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026cc810-1743-4f82-a334-9abc9484850a admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:47.141707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026cc810-1743-4f82-a334-9abc9484850a admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:47.141984 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026cc810-1743-4f82-a334-9abc9484850a admin admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:47.142760 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026cc810-1743-4f82-a334-9abc9484850a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:16:47.143083 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026cc810-1743-4f82-a334-9abc9484850a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:47.144679 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026cc810-1743-4f82-a334-9abc9484850a admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:47.156727 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 919/1822] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:16:47.160681 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1597c5-82e3-4086-8233-c2826d5bda3b None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:47.169594 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1597c5-82e3-4086-8233-c2826d5bda3b None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:47.174331 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88cee79-66f1-4a10-a7df-1c2f9fb1e92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:47.174985 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88cee79-66f1-4a10-a7df-1c2f9fb1e92b admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:47.175268 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88cee79-66f1-4a10-a7df-1c2f9fb1e92b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:47.175568 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88cee79-66f1-4a10-a7df-1c2f9fb1e92b admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:47.176265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88cee79-66f1-4a10-a7df-1c2f9fb1e92b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:47.178155 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88cee79-66f1-4a10-a7df-1c2f9fb1e92b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:47.204893 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 904/1823] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:16:47 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:16:47.208728 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026cc810-1743-4f82-a334-9abc9484850a admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:47.220745 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026cc810-1743-4f82-a334-9abc9484850a admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:47.288100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56111a30-cb1c-423c-8df6-929c49c82b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:47.288786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56111a30-cb1c-423c-8df6-929c49c82b3f admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:47.289066 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56111a30-cb1c-423c-8df6-929c49c82b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:47.289347 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56111a30-cb1c-423c-8df6-929c49c82b3f admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:47.290682 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56111a30-cb1c-423c-8df6-929c49c82b3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:47.292053 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56111a30-cb1c-423c-8df6-929c49c82b3f admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:47.320631 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 920/1824] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:16:47 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:16:47.321896 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88cee79-66f1-4a10-a7df-1c2f9fb1e92b admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:47.335473 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88cee79-66f1-4a10-a7df-1c2f9fb1e92b admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:47.341984 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b9610-c4f1-4540-b673-365081dcb355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:47.342849 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b9610-c4f1-4540-b673-365081dcb355 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:47.343344 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b9610-c4f1-4540-b673-365081dcb355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:47.343767 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b9610-c4f1-4540-b673-365081dcb355 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:47.344648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5b9610-c4f1-4540-b673-365081dcb355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:47.346532 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5b9610-c4f1-4540-b673-365081dcb355 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:47.363577 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 905/1825] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:16:47.365089 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56111a30-cb1c-423c-8df6-929c49c82b3f admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:47.375981 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56111a30-cb1c-423c-8df6-929c49c82b3f admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:47.380513 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98606d69-64c8-44b2-bef8-e186ec12414e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:47.381174 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98606d69-64c8-44b2-bef8-e186ec12414e admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:47.381485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98606d69-64c8-44b2-bef8-e186ec12414e admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:47.381753 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98606d69-64c8-44b2-bef8-e186ec12414e admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:47.382470 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98606d69-64c8-44b2-bef8-e186ec12414e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:47.384155 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98606d69-64c8-44b2-bef8-e186ec12414e admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:47.397660 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 921/1826] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:16:47.399172 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b9610-c4f1-4540-b673-365081dcb355 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:47.411633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b9610-c4f1-4540-b673-365081dcb355 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:47.416253 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a4b90-0d1f-484c-a837-6c8fa4de20ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:47.417129 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a4b90-0d1f-484c-a837-6c8fa4de20ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:47.417415 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a4b90-0d1f-484c-a837-6c8fa4de20ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:47.417722 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a4b90-0d1f-484c-a837-6c8fa4de20ce admin admin] PATH_INFO: `/v3/projects/252bde3f1d244909926f25bf1226d992/users/555a65df49c545308d67c4db68e0dbb7/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:47.431964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893a4b90-0d1f-484c-a837-6c8fa4de20ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=252bde3f1d244909926f25bf1226d992, user_id=555a65df49c545308d67c4db68e0dbb7, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:47.434625 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893a4b90-0d1f-484c-a837-6c8fa4de20ce admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:47.449261 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 906/1827] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:16:47 2022] PUT /identity/v3/projects/252bde3f1d244909926f25bf1226d992/users/555a65df49c545308d67c4db68e0dbb7/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:16:47.460906 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c82983-d8b7-49fe-8685-0b908409f7d8 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:47.461511 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c82983-d8b7-49fe-8685-0b908409f7d8 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:47.461792 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c82983-d8b7-49fe-8685-0b908409f7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:47.489243 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-41c82983-d8b7-49fe-8685-0b908409f7d8 None None] MFA Rules not processed for user `555a65df49c545308d67c4db68e0dbb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:16:47.490738 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-41c82983-d8b7-49fe-8685-0b908409f7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:47.594423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 922/1828] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:16:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:16:47.599024 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a4b90-0d1f-484c-a837-6c8fa4de20ce admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:47.620136 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:16:47.664780 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a4b90-0d1f-484c-a837-6c8fa4de20ce admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:47.669439 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f335be-e664-4a28-97c4-4c8575799621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:47.670125 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f335be-e664-4a28-97c4-4c8575799621 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:47.670407 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f335be-e664-4a28-97c4-4c8575799621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:47.670796 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f335be-e664-4a28-97c4-4c8575799621 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:47.671566 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f335be-e664-4a28-97c4-4c8575799621 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:47.673457 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f335be-e664-4a28-97c4-4c8575799621 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:47.698222 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 907/1829] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:16:47 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:16:47.702824 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c82983-d8b7-49fe-8685-0b908409f7d8 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:47.713409 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c82983-d8b7-49fe-8685-0b908409f7d8 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:47.725441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:16:47.769745 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6258b28-4e37-4058-8226-ecae158a31e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:47.770418 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6258b28-4e37-4058-8226-ecae158a31e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:47.770706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6258b28-4e37-4058-8226-ecae158a31e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:47.770993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6258b28-4e37-4058-8226-ecae158a31e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:47.771732 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6258b28-4e37-4058-8226-ecae158a31e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:47.773625 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6258b28-4e37-4058-8226-ecae158a31e2 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:47.794715 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 923/1830] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:16:47 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:16:47.798756 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f335be-e664-4a28-97c4-4c8575799621 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:47.809127 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f335be-e664-4a28-97c4-4c8575799621 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:47.814380 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3155aa7-77a4-40e2-86f6-1e6ec1ac1648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:47.815040 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3155aa7-77a4-40e2-86f6-1e6ec1ac1648 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:47.815352 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3155aa7-77a4-40e2-86f6-1e6ec1ac1648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:47.815627 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3155aa7-77a4-40e2-86f6-1e6ec1ac1648 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:47.816379 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3155aa7-77a4-40e2-86f6-1e6ec1ac1648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:47.817966 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3155aa7-77a4-40e2-86f6-1e6ec1ac1648 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:47.828800 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 908/1831] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:16:47.833155 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6258b28-4e37-4058-8226-ecae158a31e2 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:47.843492 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6258b28-4e37-4058-8226-ecae158a31e2 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:47.848306 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610a37dd-3738-4316-9b3d-183222729846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:47.849198 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a37dd-3738-4316-9b3d-183222729846 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:47.849480 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a37dd-3738-4316-9b3d-183222729846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:47.849753 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a37dd-3738-4316-9b3d-183222729846 admin admin] PATH_INFO: `/v3/projects/903e05eb64db4e898676ac47f9f2aded/users/b15a20aeec3b4e658767346e973ef220/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:47.865427 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610a37dd-3738-4316-9b3d-183222729846 admin admin] RBAC: Authorizing `identity:create_grant(project_id=903e05eb64db4e898676ac47f9f2aded, user_id=b15a20aeec3b4e658767346e973ef220, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:47.868092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610a37dd-3738-4316-9b3d-183222729846 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:47.883539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 924/1832] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:16:47 2022] PUT /identity/v3/projects/903e05eb64db4e898676ac47f9f2aded/users/b15a20aeec3b4e658767346e973ef220/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:16:47.887625 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3155aa7-77a4-40e2-86f6-1e6ec1ac1648 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:47.897415 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3155aa7-77a4-40e2-86f6-1e6ec1ac1648 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:47.953885 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3af08f5-e91e-4701-84d0-7808f11755d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:47.954548 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af08f5-e91e-4701-84d0-7808f11755d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:47.954833 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af08f5-e91e-4701-84d0-7808f11755d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:47.955112 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af08f5-e91e-4701-84d0-7808f11755d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:47.955870 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3af08f5-e91e-4701-84d0-7808f11755d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:47.957464 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3af08f5-e91e-4701-84d0-7808f11755d2 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:47.966915 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 909/1833] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:16:47.970986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610a37dd-3738-4316-9b3d-183222729846 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:47.981310 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610a37dd-3738-4316-9b3d-183222729846 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:47.986415 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad406b-6853-4af3-9fbe-36c7ebaef72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:47.987316 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad406b-6853-4af3-9fbe-36c7ebaef72a admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:47.987596 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad406b-6853-4af3-9fbe-36c7ebaef72a admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:47.987868 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad406b-6853-4af3-9fbe-36c7ebaef72a admin admin] PATH_INFO: `/v3/projects/903e05eb64db4e898676ac47f9f2aded/users/b15a20aeec3b4e658767346e973ef220/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:47.989791 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad406b-6853-4af3-9fbe-36c7ebaef72a admin admin] RBAC: Authorizing `identity:create_grant(project_id=903e05eb64db4e898676ac47f9f2aded, user_id=b15a20aeec3b4e658767346e973ef220, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:47.992465 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad406b-6853-4af3-9fbe-36c7ebaef72a admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:48.002976 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 925/1834] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:16:47 2022] PUT /identity/v3/projects/903e05eb64db4e898676ac47f9f2aded/users/b15a20aeec3b4e658767346e973ef220/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:16:48.012860 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75796eb4-9825-4a96-95a5-546a479c609f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:48.013100 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75796eb4-9825-4a96-95a5-546a479c609f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:48.013388 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75796eb4-9825-4a96-95a5-546a479c609f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:48.039528 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-75796eb4-9825-4a96-95a5-546a479c609f None None] MFA Rules not processed for user `b15a20aeec3b4e658767346e973ef220`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:16:48.040739 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-75796eb4-9825-4a96-95a5-546a479c609f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:48.107660 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 910/1835] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:16:48 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:16:48.118894 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad406b-6853-4af3-9fbe-36c7ebaef72a admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:48.120578 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-64ad406b-6853-4af3-9fbe-36c7ebaef72a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:48.177083 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad406b-6853-4af3-9fbe-36c7ebaef72a admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:48.180728 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0738a7-a0df-4dc1-a88b-8fbf1a1779dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:48.181342 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0738a7-a0df-4dc1-a88b-8fbf1a1779dd service cinder] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:48.181625 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0738a7-a0df-4dc1-a88b-8fbf1a1779dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:48.181900 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0738a7-a0df-4dc1-a88b-8fbf1a1779dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:48.237310 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0738a7-a0df-4dc1-a88b-8fbf1a1779dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:48.238739 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0738a7-a0df-4dc1-a88b-8fbf1a1779dd service cinder] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:48.247555 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 926/1836] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:16:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:16:57.196983 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75796eb4-9825-4a96-95a5-546a479c609f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:57.214076 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:16:57.261879 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75796eb4-9825-4a96-95a5-546a479c609f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:57.266576 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2767c-ec11-48ef-88b3-69c45b6abc4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:57.267516 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2767c-ec11-48ef-88b3-69c45b6abc4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:57.267804 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2767c-ec11-48ef-88b3-69c45b6abc4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:57.268082 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2767c-ec11-48ef-88b3-69c45b6abc4b admin admin] PATH_INFO: `/v3/users/555a65df49c545308d67c4db68e0dbb7` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:57.269275 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d2767c-ec11-48ef-88b3-69c45b6abc4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=555a65df49c545308d67c4db68e0dbb7)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:57.271199 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d2767c-ec11-48ef-88b3-69c45b6abc4b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:57.317731 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-56d2767c-ec11-48ef-88b3-69c45b6abc4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '555a65df49c545308d67c4db68e0dbb7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:57.324915 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-56d2767c-ec11-48ef-88b3-69c45b6abc4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '555a65df49c545308d67c4db68e0dbb7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:57.336867 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-56d2767c-ec11-48ef-88b3-69c45b6abc4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '555a65df49c545308d67c4db68e0dbb7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:57.337555 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-56d2767c-ec11-48ef-88b3-69c45b6abc4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '555a65df49c545308d67c4db68e0dbb7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:57.344950 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-56d2767c-ec11-48ef-88b3-69c45b6abc4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '555a65df49c545308d67c4db68e0dbb7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:57.350935 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 911/1837] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:16:57 2022] DELETE /identity/v3/users/555a65df49c545308d67c4db68e0dbb7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:16:57.355577 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0f480-659f-4ab1-9f0f-2ee9489d4aa1 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:57.356019 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0f480-659f-4ab1-9f0f-2ee9489d4aa1 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:57.356130 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0f480-659f-4ab1-9f0f-2ee9489d4aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:57.373431 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-d6a0f480-659f-4ab1-9f0f-2ee9489d4aa1 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:16:57.374634 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-d6a0f480-659f-4ab1-9f0f-2ee9489d4aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:57.392748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:16:57.433519 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 927/1838] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:16:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:16:57.443109 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2767c-ec11-48ef-88b3-69c45b6abc4b admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:57.444512 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-56d2767c-ec11-48ef-88b3-69c45b6abc4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:57.504480 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2767c-ec11-48ef-88b3-69c45b6abc4b admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:57.508294 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4c36a-45be-4732-a245-a4d913a11675 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:57.508977 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4c36a-45be-4732-a245-a4d913a11675 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:57.509314 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4c36a-45be-4732-a245-a4d913a11675 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:57.509609 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4c36a-45be-4732-a245-a4d913a11675 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:57.519928 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc4c36a-45be-4732-a245-a4d913a11675 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:57.521374 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc4c36a-45be-4732-a245-a4d913a11675 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:57.531839 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 912/1839] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:16:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:16:57.804870 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0f480-659f-4ab1-9f0f-2ee9489d4aa1 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:57.806455 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-d6a0f480-659f-4ab1-9f0f-2ee9489d4aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:57.819779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0f480-659f-4ab1-9f0f-2ee9489d4aa1 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:57.825711 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd568a-20df-4470-a958-a1a6c4771c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:57.827828 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd568a-20df-4470-a958-a1a6c4771c83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:57.828576 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd568a-20df-4470-a958-a1a6c4771c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:57.829378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd568a-20df-4470-a958-a1a6c4771c83 admin admin] PATH_INFO: `/v3/projects/252bde3f1d244909926f25bf1226d992` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:57.831892 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd568a-20df-4470-a958-a1a6c4771c83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=252bde3f1d244909926f25bf1226d992)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:57.835132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd568a-20df-4470-a958-a1a6c4771c83 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:57.887794 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-f3bd568a-20df-4470-a958-a1a6c4771c83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '252bde3f1d244909926f25bf1226d992', 'request_id': 'req-f3bd568a-20df-4470-a958-a1a6c4771c83'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:57.894945 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-f3bd568a-20df-4470-a958-a1a6c4771c83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '252bde3f1d244909926f25bf1226d992', 'request_id': 'req-f3bd568a-20df-4470-a958-a1a6c4771c83'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:57.904717 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-f3bd568a-20df-4470-a958-a1a6c4771c83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '252bde3f1d244909926f25bf1226d992', 'request_id': 'req-f3bd568a-20df-4470-a958-a1a6c4771c83'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:57.921691 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-f3bd568a-20df-4470-a958-a1a6c4771c83 admin admin] The token cache is being invalidate because project 252bde3f1d244909926f25bf1226d992 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:16:57.921988 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-f3bd568a-20df-4470-a958-a1a6c4771c83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:57.922722 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-f3bd568a-20df-4470-a958-a1a6c4771c83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:57.924762 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 928/1840] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:16:57 2022] DELETE /identity/v3/projects/252bde3f1d244909926f25bf1226d992 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:16:57.928547 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4c36a-45be-4732-a245-a4d913a11675 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:57.938290 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4c36a-45be-4732-a245-a4d913a11675 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:57.977847 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:16:58.007185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f3847-72bf-426d-9a81-f9237fb538ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:58.008094 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f3847-72bf-426d-9a81-f9237fb538ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:58.008379 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f3847-72bf-426d-9a81-f9237fb538ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:58.008658 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f3847-72bf-426d-9a81-f9237fb538ce admin admin] PATH_INFO: `/v3/users/b15a20aeec3b4e658767346e973ef220` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:58.009864 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452f3847-72bf-426d-9a81-f9237fb538ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=b15a20aeec3b4e658767346e973ef220)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:58.011846 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452f3847-72bf-426d-9a81-f9237fb538ce admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:58.064427 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-452f3847-72bf-426d-9a81-f9237fb538ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b15a20aeec3b4e658767346e973ef220'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:58.074022 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-452f3847-72bf-426d-9a81-f9237fb538ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b15a20aeec3b4e658767346e973ef220'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:58.087419 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-452f3847-72bf-426d-9a81-f9237fb538ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b15a20aeec3b4e658767346e973ef220'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:58.088106 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-452f3847-72bf-426d-9a81-f9237fb538ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b15a20aeec3b4e658767346e973ef220'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:58.098833 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-452f3847-72bf-426d-9a81-f9237fb538ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b15a20aeec3b4e658767346e973ef220'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:58.105808 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 913/1841] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:16:57 2022] DELETE /identity/v3/users/b15a20aeec3b4e658767346e973ef220 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:16:58.553446 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd568a-20df-4470-a958-a1a6c4771c83 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:58.554931 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-f3bd568a-20df-4470-a958-a1a6c4771c83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:58.567746 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd568a-20df-4470-a958-a1a6c4771c83 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:58.625068 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a3bf9c-ec50-4fc3-a261-56bdfb0cae75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:58.625904 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3bf9c-ec50-4fc3-a261-56bdfb0cae75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:58.626192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3bf9c-ec50-4fc3-a261-56bdfb0cae75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:58.626501 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3bf9c-ec50-4fc3-a261-56bdfb0cae75 admin admin] PATH_INFO: `/v3/projects/903e05eb64db4e898676ac47f9f2aded` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:58.627981 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a3bf9c-ec50-4fc3-a261-56bdfb0cae75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=903e05eb64db4e898676ac47f9f2aded)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:58.629900 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a3bf9c-ec50-4fc3-a261-56bdfb0cae75 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:58.678525 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-b4a3bf9c-ec50-4fc3-a261-56bdfb0cae75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '903e05eb64db4e898676ac47f9f2aded', 'request_id': 'req-b4a3bf9c-ec50-4fc3-a261-56bdfb0cae75'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:58.679652 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:16:58.685744 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-b4a3bf9c-ec50-4fc3-a261-56bdfb0cae75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '903e05eb64db4e898676ac47f9f2aded', 'request_id': 'req-b4a3bf9c-ec50-4fc3-a261-56bdfb0cae75'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:58.693740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-b4a3bf9c-ec50-4fc3-a261-56bdfb0cae75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '903e05eb64db4e898676ac47f9f2aded', 'request_id': 'req-b4a3bf9c-ec50-4fc3-a261-56bdfb0cae75'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:58.706664 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-b4a3bf9c-ec50-4fc3-a261-56bdfb0cae75 admin admin] The token cache is being invalidate because project 903e05eb64db4e898676ac47f9f2aded was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:16:58.706956 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-b4a3bf9c-ec50-4fc3-a261-56bdfb0cae75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:58.707655 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-b4a3bf9c-ec50-4fc3-a261-56bdfb0cae75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:58.710309 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 929/1842] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:16:58 2022] DELETE /identity/v3/projects/903e05eb64db4e898676ac47f9f2aded => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:16:58.715104 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd6a9a-eab7-46bb-b429-1b1729d5c53d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:58.715596 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd6a9a-eab7-46bb-b429-1b1729d5c53d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:58.715687 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd6a9a-eab7-46bb-b429-1b1729d5c53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:58.736978 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-eebd6a9a-eab7-46bb-b429-1b1729d5c53d None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:16:58.738131 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-eebd6a9a-eab7-46bb-b429-1b1729d5c53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:58.796773 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 914/1843] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:16:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:16:58.805519 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f02cf1-de7b-42d9-aaf3-cfa9b34881d0 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:58.805811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f02cf1-de7b-42d9-aaf3-cfa9b34881d0 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:58.806091 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f02cf1-de7b-42d9-aaf3-cfa9b34881d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:58.824150 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-06f02cf1-de7b-42d9-aaf3-cfa9b34881d0 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:16:58.825348 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-06f02cf1-de7b-42d9-aaf3-cfa9b34881d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:58.833963 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 930/1844] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:16:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:16:58.838098 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd6a9a-eab7-46bb-b429-1b1729d5c53d None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:58.847802 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd6a9a-eab7-46bb-b429-1b1729d5c53d None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:58.853691 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d1324-7212-47ef-aa25-cc7ecf69ed9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:58.854406 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d1324-7212-47ef-aa25-cc7ecf69ed9e admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:58.854883 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d1324-7212-47ef-aa25-cc7ecf69ed9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:58.855191 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d1324-7212-47ef-aa25-cc7ecf69ed9e admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:58.856082 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2d1324-7212-47ef-aa25-cc7ecf69ed9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:16:58.856389 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2d1324-7212-47ef-aa25-cc7ecf69ed9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:58.858153 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2d1324-7212-47ef-aa25-cc7ecf69ed9e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:58.869661 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 915/1845] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:16:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:16:58.874586 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f02cf1-de7b-42d9-aaf3-cfa9b34881d0 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:58.883964 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f02cf1-de7b-42d9-aaf3-cfa9b34881d0 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:58.889240 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cab82a-b7e3-4b3c-a3be-32b67485a856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:58.889888 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cab82a-b7e3-4b3c-a3be-32b67485a856 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:58.890183 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cab82a-b7e3-4b3c-a3be-32b67485a856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:58.890458 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cab82a-b7e3-4b3c-a3be-32b67485a856 admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:58.891164 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cab82a-b7e3-4b3c-a3be-32b67485a856 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:58.893091 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cab82a-b7e3-4b3c-a3be-32b67485a856 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:58.919062 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 931/1846] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:16:58 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:16:58.923246 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d1324-7212-47ef-aa25-cc7ecf69ed9e admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:58.934312 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d1324-7212-47ef-aa25-cc7ecf69ed9e admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:58.978187 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:16:58.995005 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd6af6a-fd00-47a2-bcf3-cc00477898ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:58.995625 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd6af6a-fd00-47a2-bcf3-cc00477898ee admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:58.995910 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd6af6a-fd00-47a2-bcf3-cc00477898ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:58.996183 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd6af6a-fd00-47a2-bcf3-cc00477898ee admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:58.996876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd6af6a-fd00-47a2-bcf3-cc00477898ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:58.999356 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd6af6a-fd00-47a2-bcf3-cc00477898ee admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:59.021527 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 916/1847] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:16:58 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:16:59.025729 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cab82a-b7e3-4b3c-a3be-32b67485a856 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:59.036552 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cab82a-b7e3-4b3c-a3be-32b67485a856 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:59.041402 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e35f312-27c5-4149-b33c-43311dc8ecce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:59.042095 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35f312-27c5-4149-b33c-43311dc8ecce admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:59.042380 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35f312-27c5-4149-b33c-43311dc8ecce admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:59.042659 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35f312-27c5-4149-b33c-43311dc8ecce admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:59.043440 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e35f312-27c5-4149-b33c-43311dc8ecce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:59.045531 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e35f312-27c5-4149-b33c-43311dc8ecce admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:59.056806 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 932/1848] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:16:59.061325 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd6af6a-fd00-47a2-bcf3-cc00477898ee admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:59.075022 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd6af6a-fd00-47a2-bcf3-cc00477898ee admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:59.079861 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98be5907-2f79-4ce4-825b-bc7634cf4c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:59.080535 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be5907-2f79-4ce4-825b-bc7634cf4c51 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:59.080820 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be5907-2f79-4ce4-825b-bc7634cf4c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:59.081098 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be5907-2f79-4ce4-825b-bc7634cf4c51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:59.081838 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98be5907-2f79-4ce4-825b-bc7634cf4c51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:59.083465 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98be5907-2f79-4ce4-825b-bc7634cf4c51 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:59.093991 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 917/1849] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:16:59.097948 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e35f312-27c5-4149-b33c-43311dc8ecce admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:59.108390 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e35f312-27c5-4149-b33c-43311dc8ecce admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:59.113129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db99e053-3405-4db1-a526-0e7eedfb478e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:59.114093 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99e053-3405-4db1-a526-0e7eedfb478e admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:59.114449 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99e053-3405-4db1-a526-0e7eedfb478e admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:59.114803 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99e053-3405-4db1-a526-0e7eedfb478e admin admin] PATH_INFO: `/v3/projects/0dd12b4e3f9142aea7fd91426cb8d683/users/655ad66e2f5a47ad85c857e3ac1f7c41/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:59.130677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db99e053-3405-4db1-a526-0e7eedfb478e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dd12b4e3f9142aea7fd91426cb8d683, user_id=655ad66e2f5a47ad85c857e3ac1f7c41, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:59.133378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db99e053-3405-4db1-a526-0e7eedfb478e admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:59.155960 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 933/1850] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:16:59 2022] PUT /identity/v3/projects/0dd12b4e3f9142aea7fd91426cb8d683/users/655ad66e2f5a47ad85c857e3ac1f7c41/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:16:59.158463 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261b085-87e4-4135-877b-db81d36b4390 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:59.158761 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261b085-87e4-4135-877b-db81d36b4390 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:59.159045 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261b085-87e4-4135-877b-db81d36b4390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:59.196590 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-7261b085-87e4-4135-877b-db81d36b4390 None None] MFA Rules not processed for user `655ad66e2f5a47ad85c857e3ac1f7c41`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:16:59.197831 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7261b085-87e4-4135-877b-db81d36b4390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:59.265330 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 918/1851] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:16:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:16:59.278304 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db99e053-3405-4db1-a526-0e7eedfb478e admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:59.280607 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-db99e053-3405-4db1-a526-0e7eedfb478e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:59.337331 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db99e053-3405-4db1-a526-0e7eedfb478e admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:59.341086 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c401f8-d449-4496-a346-202f8a0a6394 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:59.341771 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c401f8-d449-4496-a346-202f8a0a6394 service cinder] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:59.342025 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c401f8-d449-4496-a346-202f8a0a6394 service cinder] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:59.342434 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c401f8-d449-4496-a346-202f8a0a6394 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:59.351223 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c401f8-d449-4496-a346-202f8a0a6394 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:59.352713 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c401f8-d449-4496-a346-202f8a0a6394 service cinder] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:59.362181 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 934/1852] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:16:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:16:59.761408 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261b085-87e4-4135-877b-db81d36b4390 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:16:59.763768 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7261b085-87e4-4135-877b-db81d36b4390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:16:59.831408 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261b085-87e4-4135-877b-db81d36b4390 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:16:59.835927 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf15116-a5d0-4f5f-a136-1ee8a0b2cc71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:16:59.836811 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf15116-a5d0-4f5f-a136-1ee8a0b2cc71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:59.837091 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf15116-a5d0-4f5f-a136-1ee8a0b2cc71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:59.837376 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf15116-a5d0-4f5f-a136-1ee8a0b2cc71 admin admin] PATH_INFO: `/v3/users/63a7efe3bf244e07b73a8d57acdc234e` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:59.838747 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf15116-a5d0-4f5f-a136-1ee8a0b2cc71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63a7efe3bf244e07b73a8d57acdc234e)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:16:59.840762 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf15116-a5d0-4f5f-a136-1ee8a0b2cc71 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:16:59.885017 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ebf15116-a5d0-4f5f-a136-1ee8a0b2cc71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63a7efe3bf244e07b73a8d57acdc234e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:59.892989 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ebf15116-a5d0-4f5f-a136-1ee8a0b2cc71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63a7efe3bf244e07b73a8d57acdc234e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:59.894439 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:16:59.909138 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ebf15116-a5d0-4f5f-a136-1ee8a0b2cc71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63a7efe3bf244e07b73a8d57acdc234e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:59.909796 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ebf15116-a5d0-4f5f-a136-1ee8a0b2cc71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63a7efe3bf244e07b73a8d57acdc234e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:59.916710 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ebf15116-a5d0-4f5f-a136-1ee8a0b2cc71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63a7efe3bf244e07b73a8d57acdc234e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:16:59.922582 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 919/1853] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:16:59 2022] DELETE /identity/v3/users/63a7efe3bf244e07b73a8d57acdc234e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:16:59.927068 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52952dae-5b3a-4f2b-999b-0a14bf34269a None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:16:59.927588 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52952dae-5b3a-4f2b-999b-0a14bf34269a None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:16:59.927681 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52952dae-5b3a-4f2b-999b-0a14bf34269a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:16:59.944190 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-52952dae-5b3a-4f2b-999b-0a14bf34269a None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:16:59.945552 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-52952dae-5b3a-4f2b-999b-0a14bf34269a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:00.003316 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 935/1854] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:16:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:17:00.013307 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf15116-a5d0-4f5f-a136-1ee8a0b2cc71 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:00.014704 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ebf15116-a5d0-4f5f-a136-1ee8a0b2cc71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:00.073591 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf15116-a5d0-4f5f-a136-1ee8a0b2cc71 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:00.079487 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a43004-04f2-4f3f-9cb4-a8861c0ce546 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:00.080174 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a43004-04f2-4f3f-9cb4-a8861c0ce546 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:00.080462 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a43004-04f2-4f3f-9cb4-a8861c0ce546 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:00.080739 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a43004-04f2-4f3f-9cb4-a8861c0ce546 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:00.091953 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a43004-04f2-4f3f-9cb4-a8861c0ce546 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:00.093388 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a43004-04f2-4f3f-9cb4-a8861c0ce546 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:00.103691 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 920/1855] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:17:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:17:01.270474 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52952dae-5b3a-4f2b-999b-0a14bf34269a None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:01.272510 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-52952dae-5b3a-4f2b-999b-0a14bf34269a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:01.285463 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52952dae-5b3a-4f2b-999b-0a14bf34269a None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:01.291060 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba758e6-9668-4eac-b3d2-bd8b685dc364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:01.292139 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba758e6-9668-4eac-b3d2-bd8b685dc364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:01.292487 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba758e6-9668-4eac-b3d2-bd8b685dc364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:01.292834 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba758e6-9668-4eac-b3d2-bd8b685dc364 admin admin] PATH_INFO: `/v3/projects/6f708b764aed4b748bf934acc4e139eb` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:01.294785 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba758e6-9668-4eac-b3d2-bd8b685dc364 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f708b764aed4b748bf934acc4e139eb)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:01.300176 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba758e6-9668-4eac-b3d2-bd8b685dc364 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:01.312596 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:01.355242 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-1ba758e6-9668-4eac-b3d2-bd8b685dc364 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f708b764aed4b748bf934acc4e139eb', 'request_id': 'req-1ba758e6-9668-4eac-b3d2-bd8b685dc364'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:01.364174 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-1ba758e6-9668-4eac-b3d2-bd8b685dc364 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f708b764aed4b748bf934acc4e139eb', 'request_id': 'req-1ba758e6-9668-4eac-b3d2-bd8b685dc364'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:01.374420 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-1ba758e6-9668-4eac-b3d2-bd8b685dc364 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f708b764aed4b748bf934acc4e139eb', 'request_id': 'req-1ba758e6-9668-4eac-b3d2-bd8b685dc364'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:01.386789 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-1ba758e6-9668-4eac-b3d2-bd8b685dc364 admin admin] The token cache is being invalidate because project 6f708b764aed4b748bf934acc4e139eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:17:01.387096 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-1ba758e6-9668-4eac-b3d2-bd8b685dc364 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:01.389515 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-1ba758e6-9668-4eac-b3d2-bd8b685dc364 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:01.391675 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 936/1856] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:17:01 2022] DELETE /identity/v3/projects/6f708b764aed4b748bf934acc4e139eb => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:01.396217 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a43004-04f2-4f3f-9cb4-a8861c0ce546 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:01.406116 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a43004-04f2-4f3f-9cb4-a8861c0ce546 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:01.475907 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bffa3c-c6e1-4181-973a-4a7e390f38bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:01.476796 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bffa3c-c6e1-4181-973a-4a7e390f38bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:01.477079 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bffa3c-c6e1-4181-973a-4a7e390f38bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:01.477355 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bffa3c-c6e1-4181-973a-4a7e390f38bb admin admin] PATH_INFO: `/v3/users/08cc586421ec4002ba94bb753ade9234` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:01.478555 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bffa3c-c6e1-4181-973a-4a7e390f38bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=08cc586421ec4002ba94bb753ade9234)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:01.480587 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bffa3c-c6e1-4181-973a-4a7e390f38bb admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:01.556112 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-44bffa3c-c6e1-4181-973a-4a7e390f38bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08cc586421ec4002ba94bb753ade9234'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:01.565943 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-44bffa3c-c6e1-4181-973a-4a7e390f38bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08cc586421ec4002ba94bb753ade9234'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:01.578847 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-44bffa3c-c6e1-4181-973a-4a7e390f38bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08cc586421ec4002ba94bb753ade9234'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:01.579500 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-44bffa3c-c6e1-4181-973a-4a7e390f38bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08cc586421ec4002ba94bb753ade9234'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:01.585082 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:01.588194 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-44bffa3c-c6e1-4181-973a-4a7e390f38bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08cc586421ec4002ba94bb753ade9234'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:01.598104 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 921/1857] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:17:01 2022] DELETE /identity/v3/users/08cc586421ec4002ba94bb753ade9234 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:01.931958 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba758e6-9668-4eac-b3d2-bd8b685dc364 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:01.933474 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-1ba758e6-9668-4eac-b3d2-bd8b685dc364 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:01.947246 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba758e6-9668-4eac-b3d2-bd8b685dc364 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:02.005117 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e444659-c0d6-4698-afd0-b11b2e6a921c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:02.005995 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e444659-c0d6-4698-afd0-b11b2e6a921c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:02.006284 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e444659-c0d6-4698-afd0-b11b2e6a921c admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:02.006565 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e444659-c0d6-4698-afd0-b11b2e6a921c admin admin] PATH_INFO: `/v3/projects/9e2587f6bc2e4012b1e2ee6092b62041` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:02.007794 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e444659-c0d6-4698-afd0-b11b2e6a921c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e2587f6bc2e4012b1e2ee6092b62041)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:02.009689 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e444659-c0d6-4698-afd0-b11b2e6a921c admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:02.061340 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-8e444659-c0d6-4698-afd0-b11b2e6a921c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e2587f6bc2e4012b1e2ee6092b62041', 'request_id': 'req-8e444659-c0d6-4698-afd0-b11b2e6a921c'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:02.069261 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-8e444659-c0d6-4698-afd0-b11b2e6a921c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e2587f6bc2e4012b1e2ee6092b62041', 'request_id': 'req-8e444659-c0d6-4698-afd0-b11b2e6a921c'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:02.078679 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-8e444659-c0d6-4698-afd0-b11b2e6a921c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e2587f6bc2e4012b1e2ee6092b62041', 'request_id': 'req-8e444659-c0d6-4698-afd0-b11b2e6a921c'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:02.090485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-8e444659-c0d6-4698-afd0-b11b2e6a921c admin admin] The token cache is being invalidate because project 9e2587f6bc2e4012b1e2ee6092b62041 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:17:02.090939 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-8e444659-c0d6-4698-afd0-b11b2e6a921c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:02.092127 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-8e444659-c0d6-4698-afd0-b11b2e6a921c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:02.093917 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 937/1858] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:17:01 2022] DELETE /identity/v3/projects/9e2587f6bc2e4012b1e2ee6092b62041 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:02.098578 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2a0c9-dcf6-4891-93c3-a2893139c618 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:02.098872 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2a0c9-dcf6-4891-93c3-a2893139c618 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:02.099150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2a0c9-dcf6-4891-93c3-a2893139c618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:02.125566 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-e7a2a0c9-dcf6-4891-93c3-a2893139c618 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:02.126774 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e7a2a0c9-dcf6-4891-93c3-a2893139c618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:02.188096 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 922/1859] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:17:02.195749 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edef10b-01d9-4510-a0c5-708623325967 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:02.196068 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edef10b-01d9-4510-a0c5-708623325967 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:02.196348 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edef10b-01d9-4510-a0c5-708623325967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:02.214709 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-6edef10b-01d9-4510-a0c5-708623325967 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:02.216794 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-6edef10b-01d9-4510-a0c5-708623325967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:02.224610 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 938/1860] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:17:02.228847 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2a0c9-dcf6-4891-93c3-a2893139c618 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:02.238310 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2a0c9-dcf6-4891-93c3-a2893139c618 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:02.242767 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af6417-2bca-422a-847c-7b16b94601dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:02.243447 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af6417-2bca-422a-847c-7b16b94601dd admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:02.243732 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af6417-2bca-422a-847c-7b16b94601dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:02.244010 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af6417-2bca-422a-847c-7b16b94601dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:02.244733 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4af6417-2bca-422a-847c-7b16b94601dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:17:02.245041 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4af6417-2bca-422a-847c-7b16b94601dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:02.246647 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4af6417-2bca-422a-847c-7b16b94601dd admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:02.258098 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 923/1861] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:17:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:17:02.262360 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edef10b-01d9-4510-a0c5-708623325967 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:02.272234 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edef10b-01d9-4510-a0c5-708623325967 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:02.276869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8827ed60-0b31-422e-8c74-1e72ea7bca71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:02.277825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827ed60-0b31-422e-8c74-1e72ea7bca71 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:02.278095 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827ed60-0b31-422e-8c74-1e72ea7bca71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:02.279067 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827ed60-0b31-422e-8c74-1e72ea7bca71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:02.279808 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8827ed60-0b31-422e-8c74-1e72ea7bca71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:02.281748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8827ed60-0b31-422e-8c74-1e72ea7bca71 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:02.306324 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 939/1862] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:17:02 2022] POST /identity/v3/projects => generated 377 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:17:02.310327 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af6417-2bca-422a-847c-7b16b94601dd admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:02.319932 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af6417-2bca-422a-847c-7b16b94601dd admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:02.375767 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faec35a-ff30-4790-8f03-65f46a0f7140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:02.376416 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faec35a-ff30-4790-8f03-65f46a0f7140 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:02.376700 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faec35a-ff30-4790-8f03-65f46a0f7140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:02.376977 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faec35a-ff30-4790-8f03-65f46a0f7140 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:02.377662 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faec35a-ff30-4790-8f03-65f46a0f7140 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:02.379589 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faec35a-ff30-4790-8f03-65f46a0f7140 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:02.398748 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 924/1863] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:17:02 2022] POST /identity/v3/users => generated 289 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:17:02.403059 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8827ed60-0b31-422e-8c74-1e72ea7bca71 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:02.408615 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:02.414696 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8827ed60-0b31-422e-8c74-1e72ea7bca71 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:02.420018 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82420ab1-f68e-42a6-96cd-16e1a35471ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:02.420672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82420ab1-f68e-42a6-96cd-16e1a35471ad admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:02.420957 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82420ab1-f68e-42a6-96cd-16e1a35471ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:02.421237 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82420ab1-f68e-42a6-96cd-16e1a35471ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:02.421963 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82420ab1-f68e-42a6-96cd-16e1a35471ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:02.423571 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82420ab1-f68e-42a6-96cd-16e1a35471ad admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:02.433771 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 940/1864] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:17:02.438030 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faec35a-ff30-4790-8f03-65f46a0f7140 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:02.447917 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faec35a-ff30-4790-8f03-65f46a0f7140 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:02.453004 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af06e5be-5f94-4d69-8e61-030c26b932f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:02.453649 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06e5be-5f94-4d69-8e61-030c26b932f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:02.453932 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06e5be-5f94-4d69-8e61-030c26b932f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:02.454208 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06e5be-5f94-4d69-8e61-030c26b932f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:02.454935 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af06e5be-5f94-4d69-8e61-030c26b932f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:02.456540 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af06e5be-5f94-4d69-8e61-030c26b932f3 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:02.466153 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 925/1865] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:17:02.470141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82420ab1-f68e-42a6-96cd-16e1a35471ad admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:02.480148 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82420ab1-f68e-42a6-96cd-16e1a35471ad admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:02.484775 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65835144-9b41-44f5-a676-00f50f599b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:02.485656 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65835144-9b41-44f5-a676-00f50f599b54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:02.485943 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65835144-9b41-44f5-a676-00f50f599b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:02.486225 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65835144-9b41-44f5-a676-00f50f599b54 admin admin] PATH_INFO: `/v3/projects/90e7d35a6d114ab7b793b452e40904d5/users/95d940dcc628403295c125352a2c95e4/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:02.501119 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65835144-9b41-44f5-a676-00f50f599b54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90e7d35a6d114ab7b793b452e40904d5, user_id=95d940dcc628403295c125352a2c95e4, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:02.503789 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65835144-9b41-44f5-a676-00f50f599b54 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:02.517726 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 941/1866] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:17:02 2022] PUT /identity/v3/projects/90e7d35a6d114ab7b793b452e40904d5/users/95d940dcc628403295c125352a2c95e4/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:17:02.528172 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e5e8a-c67d-4c38-835d-59a865fbe22a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:02.528497 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e5e8a-c67d-4c38-835d-59a865fbe22a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:02.528773 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e5e8a-c67d-4c38-835d-59a865fbe22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:02.554951 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-e95e5e8a-c67d-4c38-835d-59a865fbe22a None None] MFA Rules not processed for user `95d940dcc628403295c125352a2c95e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:02.556182 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e95e5e8a-c67d-4c38-835d-59a865fbe22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:02.603683 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:02.623551 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 926/1867] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:17:02.628022 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65835144-9b41-44f5-a676-00f50f599b54 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:02.706518 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65835144-9b41-44f5-a676-00f50f599b54 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:02.711171 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a2b36d-ae4a-42c0-86aa-d5b980b0f639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:02.711871 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2b36d-ae4a-42c0-86aa-d5b980b0f639 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:02.712156 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2b36d-ae4a-42c0-86aa-d5b980b0f639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:02.712433 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2b36d-ae4a-42c0-86aa-d5b980b0f639 admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:02.713114 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a2b36d-ae4a-42c0-86aa-d5b980b0f639 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:02.714999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a2b36d-ae4a-42c0-86aa-d5b980b0f639 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:02.740401 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 942/1868] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:17:02 2022] POST /identity/v3/projects => generated 379 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:17:02.744636 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e5e8a-c67d-4c38-835d-59a865fbe22a None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:02.757172 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e5e8a-c67d-4c38-835d-59a865fbe22a None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:02.820659 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb95e4-6209-46ff-a9b0-e58444a40776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:02.821333 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb95e4-6209-46ff-a9b0-e58444a40776 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:02.821616 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb95e4-6209-46ff-a9b0-e58444a40776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:02.821891 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb95e4-6209-46ff-a9b0-e58444a40776 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:02.822573 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb95e4-6209-46ff-a9b0-e58444a40776 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:02.824493 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb95e4-6209-46ff-a9b0-e58444a40776 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:02.844741 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 927/1869] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:17:02 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:17:02.848523 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a2b36d-ae4a-42c0-86aa-d5b980b0f639 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:02.859138 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a2b36d-ae4a-42c0-86aa-d5b980b0f639 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:02.863703 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b4bd14-c642-456c-87a6-fba156aebc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:02.864353 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4bd14-c642-456c-87a6-fba156aebc95 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:02.864638 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4bd14-c642-456c-87a6-fba156aebc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:02.864912 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4bd14-c642-456c-87a6-fba156aebc95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:02.865656 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b4bd14-c642-456c-87a6-fba156aebc95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:02.867219 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b4bd14-c642-456c-87a6-fba156aebc95 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:02.878327 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 943/1870] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:17:02.882155 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb95e4-6209-46ff-a9b0-e58444a40776 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:02.892084 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb95e4-6209-46ff-a9b0-e58444a40776 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:02.896658 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c130f548-84af-41d5-9234-0506f567098e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:02.897319 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130f548-84af-41d5-9234-0506f567098e admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:02.897609 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130f548-84af-41d5-9234-0506f567098e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:02.897892 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130f548-84af-41d5-9234-0506f567098e admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:02.898630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c130f548-84af-41d5-9234-0506f567098e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:02.900273 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c130f548-84af-41d5-9234-0506f567098e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:02.910625 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 928/1871] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:17:02.914762 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b4bd14-c642-456c-87a6-fba156aebc95 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:02.925249 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b4bd14-c642-456c-87a6-fba156aebc95 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:02.930004 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609907f8-77f5-41f4-9a25-d66c09b4bfb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:02.930836 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609907f8-77f5-41f4-9a25-d66c09b4bfb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:02.931176 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609907f8-77f5-41f4-9a25-d66c09b4bfb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:02.931492 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609907f8-77f5-41f4-9a25-d66c09b4bfb3 admin admin] PATH_INFO: `/v3/projects/a18b838e6b384a8a923d46960e552f2a/users/e26549614eff43b2a69e2a2cfac9a4b6/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:02.947569 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609907f8-77f5-41f4-9a25-d66c09b4bfb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a18b838e6b384a8a923d46960e552f2a, user_id=e26549614eff43b2a69e2a2cfac9a4b6, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:02.950218 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609907f8-77f5-41f4-9a25-d66c09b4bfb3 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:02.964386 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 944/1872] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:17:02 2022] PUT /identity/v3/projects/a18b838e6b384a8a923d46960e552f2a/users/e26549614eff43b2a69e2a2cfac9a4b6/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:02.975558 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4d635-fd71-4295-bb5f-9ca965b5b333 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:02.975910 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4d635-fd71-4295-bb5f-9ca965b5b333 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:02.976196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4d635-fd71-4295-bb5f-9ca965b5b333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:03.011823 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-d5b4d635-fd71-4295-bb5f-9ca965b5b333 None None] MFA Rules not processed for user `e26549614eff43b2a69e2a2cfac9a4b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:03.013085 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d5b4d635-fd71-4295-bb5f-9ca965b5b333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:03.083649 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 929/1873] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:17:03.088200 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609907f8-77f5-41f4-9a25-d66c09b4bfb3 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:03.154144 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609907f8-77f5-41f4-9a25-d66c09b4bfb3 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:03.159060 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b14e9a4-eba4-4b1c-9397-a23587539b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:03.159814 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14e9a4-eba4-4b1c-9397-a23587539b1c admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:03.160102 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14e9a4-eba4-4b1c-9397-a23587539b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:03.160378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14e9a4-eba4-4b1c-9397-a23587539b1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:03.161081 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b14e9a4-eba4-4b1c-9397-a23587539b1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:03.162957 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b14e9a4-eba4-4b1c-9397-a23587539b1c admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:03.189709 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 945/1874] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:17:03 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:17:03.193761 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4d635-fd71-4295-bb5f-9ca965b5b333 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:03.205220 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4d635-fd71-4295-bb5f-9ca965b5b333 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:03.266976 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:03.274765 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de21c48-46a3-4857-b87c-b947f2835e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:03.275772 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de21c48-46a3-4857-b87c-b947f2835e46 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:03.275978 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de21c48-46a3-4857-b87c-b947f2835e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:03.276247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de21c48-46a3-4857-b87c-b947f2835e46 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:03.277182 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de21c48-46a3-4857-b87c-b947f2835e46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:03.280051 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de21c48-46a3-4857-b87c-b947f2835e46 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:03.299639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 930/1875] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:17:03 2022] POST /identity/v3/users => generated 296 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:17:03.303703 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b14e9a4-eba4-4b1c-9397-a23587539b1c admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:03.309152 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:03.316340 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b14e9a4-eba4-4b1c-9397-a23587539b1c admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:03.321555 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc6da2b-90a1-41c0-bc4e-ca06c45dce2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:03.322348 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc6da2b-90a1-41c0-bc4e-ca06c45dce2e admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:03.322699 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc6da2b-90a1-41c0-bc4e-ca06c45dce2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:03.323043 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc6da2b-90a1-41c0-bc4e-ca06c45dce2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:03.323921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc6da2b-90a1-41c0-bc4e-ca06c45dce2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:03.326120 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc6da2b-90a1-41c0-bc4e-ca06c45dce2e admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:03.336789 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 946/1876] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:17:03.341030 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de21c48-46a3-4857-b87c-b947f2835e46 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:03.351619 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de21c48-46a3-4857-b87c-b947f2835e46 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:03.356207 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ab352-ec06-4197-a66c-e990071875e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:03.357099 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ab352-ec06-4197-a66c-e990071875e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:03.357382 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ab352-ec06-4197-a66c-e990071875e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:03.357654 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ab352-ec06-4197-a66c-e990071875e5 admin admin] PATH_INFO: `/v3/projects/2c7ff9568fcd4951892e77dcc0f2b9c1/users/61c162da02a64cb9aa81fe5adc51d1ff/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:03.371781 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ab352-ec06-4197-a66c-e990071875e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c7ff9568fcd4951892e77dcc0f2b9c1, user_id=61c162da02a64cb9aa81fe5adc51d1ff, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:03.374925 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ab352-ec06-4197-a66c-e990071875e5 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:03.387415 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 931/1877] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:17:03 2022] PUT /identity/v3/projects/2c7ff9568fcd4951892e77dcc0f2b9c1/users/61c162da02a64cb9aa81fe5adc51d1ff/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:03.391879 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc6da2b-90a1-41c0-bc4e-ca06c45dce2e admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:03.401641 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc6da2b-90a1-41c0-bc4e-ca06c45dce2e admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:03.456609 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b01fc-60c8-4e21-be03-684f5ecd602c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:03.457208 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b01fc-60c8-4e21-be03-684f5ecd602c admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:03.457500 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b01fc-60c8-4e21-be03-684f5ecd602c admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:03.457783 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b01fc-60c8-4e21-be03-684f5ecd602c admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:03.458521 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9b01fc-60c8-4e21-be03-684f5ecd602c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:03.460184 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9b01fc-60c8-4e21-be03-684f5ecd602c admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:03.470507 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 947/1878] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:17:03.474620 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ab352-ec06-4197-a66c-e990071875e5 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:03.485190 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ab352-ec06-4197-a66c-e990071875e5 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:03.489714 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26c3671-b27d-499f-8301-6bbcac189d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:03.490579 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c3671-b27d-499f-8301-6bbcac189d0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:03.490862 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c3671-b27d-499f-8301-6bbcac189d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:03.491147 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c3671-b27d-499f-8301-6bbcac189d0b admin admin] PATH_INFO: `/v3/projects/2c7ff9568fcd4951892e77dcc0f2b9c1/users/61c162da02a64cb9aa81fe5adc51d1ff/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:03.493100 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26c3671-b27d-499f-8301-6bbcac189d0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c7ff9568fcd4951892e77dcc0f2b9c1, user_id=61c162da02a64cb9aa81fe5adc51d1ff, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:03.495756 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26c3671-b27d-499f-8301-6bbcac189d0b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:03.505716 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 932/1879] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:17:03 2022] PUT /identity/v3/projects/2c7ff9568fcd4951892e77dcc0f2b9c1/users/61c162da02a64cb9aa81fe5adc51d1ff/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:03.515845 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd08d6-8853-4c0c-9973-e449c701b10f None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:03.516140 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd08d6-8853-4c0c-9973-e449c701b10f None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:03.516421 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd08d6-8853-4c0c-9973-e449c701b10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:03.542453 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-8cbd08d6-8853-4c0c-9973-e449c701b10f None None] MFA Rules not processed for user `61c162da02a64cb9aa81fe5adc51d1ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:03.543738 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-8cbd08d6-8853-4c0c-9973-e449c701b10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:03.609009 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 948/1880] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:03 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:17:03.619938 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26c3671-b27d-499f-8301-6bbcac189d0b admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:03.621649 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f26c3671-b27d-499f-8301-6bbcac189d0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:03.679145 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26c3671-b27d-499f-8301-6bbcac189d0b admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:03.683529 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d03c4f-9c77-4db3-8267-797b9672b95f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:03.684177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d03c4f-9c77-4db3-8267-797b9672b95f service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:03.684464 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d03c4f-9c77-4db3-8267-797b9672b95f service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:03.684745 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d03c4f-9c77-4db3-8267-797b9672b95f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:03.693776 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d03c4f-9c77-4db3-8267-797b9672b95f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:03.695213 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d03c4f-9c77-4db3-8267-797b9672b95f service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:03.705477 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 933/1881] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:17:03 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:17:03.745643 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd08d6-8853-4c0c-9973-e449c701b10f None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:03.747696 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-8cbd08d6-8853-4c0c-9973-e449c701b10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:03.759514 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd08d6-8853-4c0c-9973-e449c701b10f None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:03.763747 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3cfde1-c6af-4f86-a95a-6f56bdbb9b98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:03.764498 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3cfde1-c6af-4f86-a95a-6f56bdbb9b98 service cinder] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:03.764854 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3cfde1-c6af-4f86-a95a-6f56bdbb9b98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:03.765203 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3cfde1-c6af-4f86-a95a-6f56bdbb9b98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:03.851065 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3cfde1-c6af-4f86-a95a-6f56bdbb9b98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:03.852636 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3cfde1-c6af-4f86-a95a-6f56bdbb9b98 service cinder] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:03.874781 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 949/1882] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:17:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:17:10.132801 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d03c4f-9c77-4db3-8267-797b9672b95f service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:10.134244 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c6d03c4f-9c77-4db3-8267-797b9672b95f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:10.217394 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d03c4f-9c77-4db3-8267-797b9672b95f service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:10.223092 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d52625-86e0-451b-9273-e2b118b53e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:10.224012 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d52625-86e0-451b-9273-e2b118b53e0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:10.224302 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d52625-86e0-451b-9273-e2b118b53e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:10.224588 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d52625-86e0-451b-9273-e2b118b53e0a admin admin] PATH_INFO: `/v3/users/655ad66e2f5a47ad85c857e3ac1f7c41` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:10.225819 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d52625-86e0-451b-9273-e2b118b53e0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=655ad66e2f5a47ad85c857e3ac1f7c41)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:10.227806 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d52625-86e0-451b-9273-e2b118b53e0a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:10.285236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f9d52625-86e0-451b-9273-e2b118b53e0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '655ad66e2f5a47ad85c857e3ac1f7c41'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:10.297496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f9d52625-86e0-451b-9273-e2b118b53e0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '655ad66e2f5a47ad85c857e3ac1f7c41'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:10.298967 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=86,keystone:INSERT=6,keystone:DELETE=8 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:10.317772 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f9d52625-86e0-451b-9273-e2b118b53e0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '655ad66e2f5a47ad85c857e3ac1f7c41'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:10.318490 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f9d52625-86e0-451b-9273-e2b118b53e0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '655ad66e2f5a47ad85c857e3ac1f7c41'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:10.326139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f9d52625-86e0-451b-9273-e2b118b53e0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '655ad66e2f5a47ad85c857e3ac1f7c41'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:10.332124 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 934/1883] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:17:10 2022] DELETE /identity/v3/users/655ad66e2f5a47ad85c857e3ac1f7c41 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:17:10.336676 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267b0e5-3e42-4872-a8bf-30853fd1be09 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:10.337204 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267b0e5-3e42-4872-a8bf-30853fd1be09 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:10.337378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267b0e5-3e42-4872-a8bf-30853fd1be09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:10.354633 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-0267b0e5-3e42-4872-a8bf-30853fd1be09 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:10.356138 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-0267b0e5-3e42-4872-a8bf-30853fd1be09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:10.362326 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:10.416874 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 950/1884] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:17:10.426802 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d52625-86e0-451b-9273-e2b118b53e0a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:10.428191 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f9d52625-86e0-451b-9273-e2b118b53e0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:10.482665 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d52625-86e0-451b-9273-e2b118b53e0a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:10.486233 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd3190-982f-41ab-b525-deff87edf1ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:10.486867 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd3190-982f-41ab-b525-deff87edf1ae service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:10.487149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd3190-982f-41ab-b525-deff87edf1ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:10.487468 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd3190-982f-41ab-b525-deff87edf1ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:10.496241 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd3190-982f-41ab-b525-deff87edf1ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:10.498218 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd3190-982f-41ab-b525-deff87edf1ae service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:10.507401 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 935/1885] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:17:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:17:10.804529 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0267b0e5-3e42-4872-a8bf-30853fd1be09 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:10.805983 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-0267b0e5-3e42-4872-a8bf-30853fd1be09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:10.820027 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0267b0e5-3e42-4872-a8bf-30853fd1be09 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:10.830075 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed10dd8-c1dc-4528-b527-5d46ffb64e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:10.830963 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed10dd8-c1dc-4528-b527-5d46ffb64e5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:10.831254 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed10dd8-c1dc-4528-b527-5d46ffb64e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:10.831571 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed10dd8-c1dc-4528-b527-5d46ffb64e5a admin admin] PATH_INFO: `/v3/projects/0dd12b4e3f9142aea7fd91426cb8d683` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:10.832766 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed10dd8-c1dc-4528-b527-5d46ffb64e5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dd12b4e3f9142aea7fd91426cb8d683)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:10.834706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed10dd8-c1dc-4528-b527-5d46ffb64e5a admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:10.896269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0ed10dd8-c1dc-4528-b527-5d46ffb64e5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dd12b4e3f9142aea7fd91426cb8d683', 'request_id': 'req-0ed10dd8-c1dc-4528-b527-5d46ffb64e5a'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:10.903940 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0ed10dd8-c1dc-4528-b527-5d46ffb64e5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dd12b4e3f9142aea7fd91426cb8d683', 'request_id': 'req-0ed10dd8-c1dc-4528-b527-5d46ffb64e5a'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:10.913153 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0ed10dd8-c1dc-4528-b527-5d46ffb64e5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dd12b4e3f9142aea7fd91426cb8d683', 'request_id': 'req-0ed10dd8-c1dc-4528-b527-5d46ffb64e5a'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:10.936525 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0ed10dd8-c1dc-4528-b527-5d46ffb64e5a admin admin] The token cache is being invalidate because project 0dd12b4e3f9142aea7fd91426cb8d683 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:17:10.936852 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0ed10dd8-c1dc-4528-b527-5d46ffb64e5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:10.937512 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0ed10dd8-c1dc-4528-b527-5d46ffb64e5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:10.939070 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 951/1886] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:17:10 2022] DELETE /identity/v3/projects/0dd12b4e3f9142aea7fd91426cb8d683 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:10.947285 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7b003-55f2-461c-8626-71ee80ab0131 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:10.947661 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7b003-55f2-461c-8626-71ee80ab0131 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:10.947877 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7b003-55f2-461c-8626-71ee80ab0131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:10.972291 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-eca7b003-55f2-461c-8626-71ee80ab0131 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:10.973625 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-eca7b003-55f2-461c-8626-71ee80ab0131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:11.050638 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 936/1887] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:17:11.058747 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c0dda-9c64-435d-8cca-3e37a826f1e6 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:11.059118 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c0dda-9c64-435d-8cca-3e37a826f1e6 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:11.060271 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c0dda-9c64-435d-8cca-3e37a826f1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:11.085367 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-e71c0dda-9c64-435d-8cca-3e37a826f1e6 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:11.086775 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-e71c0dda-9c64-435d-8cca-3e37a826f1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:11.105060 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 952/1888] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:17:11.109345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7b003-55f2-461c-8626-71ee80ab0131 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:11.119277 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7b003-55f2-461c-8626-71ee80ab0131 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:11.125209 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2f1bf-74d6-4adb-913a-223cde90d95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:11.125808 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2f1bf-74d6-4adb-913a-223cde90d95f admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:11.126213 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2f1bf-74d6-4adb-913a-223cde90d95f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:11.126494 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2f1bf-74d6-4adb-913a-223cde90d95f admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:11.127232 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2f1bf-74d6-4adb-913a-223cde90d95f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:17:11.128006 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2f1bf-74d6-4adb-913a-223cde90d95f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:11.129170 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2f1bf-74d6-4adb-913a-223cde90d95f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:11.142069 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 937/1889] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:17:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:17:11.146883 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c0dda-9c64-435d-8cca-3e37a826f1e6 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:11.157524 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c0dda-9c64-435d-8cca-3e37a826f1e6 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:11.162375 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464b0df-36ed-4b2e-937c-b2853b6108e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:11.176357 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464b0df-36ed-4b2e-937c-b2853b6108e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:11.176825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464b0df-36ed-4b2e-937c-b2853b6108e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:11.177120 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464b0df-36ed-4b2e-937c-b2853b6108e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:11.178174 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464b0df-36ed-4b2e-937c-b2853b6108e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:11.182190 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464b0df-36ed-4b2e-937c-b2853b6108e3 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:11.212500 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 953/1890] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:17:11 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:17:11.217549 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2f1bf-74d6-4adb-913a-223cde90d95f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:11.231987 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2f1bf-74d6-4adb-913a-223cde90d95f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:11.305650 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10237950-fee3-41b0-a018-4b3ccd0b38d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:11.306326 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10237950-fee3-41b0-a018-4b3ccd0b38d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:11.306616 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10237950-fee3-41b0-a018-4b3ccd0b38d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:11.306903 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10237950-fee3-41b0-a018-4b3ccd0b38d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:11.307643 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10237950-fee3-41b0-a018-4b3ccd0b38d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:11.310094 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10237950-fee3-41b0-a018-4b3ccd0b38d0 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:11.318244 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:11.337896 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 938/1891] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:17:11 2022] POST /identity/v3/users => generated 289 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:17:11.342429 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464b0df-36ed-4b2e-937c-b2853b6108e3 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:11.353957 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464b0df-36ed-4b2e-937c-b2853b6108e3 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:11.359697 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da30b00-6cc2-46d6-a788-6c2ff4cf9bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:11.360388 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da30b00-6cc2-46d6-a788-6c2ff4cf9bb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:11.360721 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da30b00-6cc2-46d6-a788-6c2ff4cf9bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:11.361065 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da30b00-6cc2-46d6-a788-6c2ff4cf9bb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:11.361865 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da30b00-6cc2-46d6-a788-6c2ff4cf9bb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:11.363611 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da30b00-6cc2-46d6-a788-6c2ff4cf9bb8 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:11.377444 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 954/1892] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:17:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:17:11.381377 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10237950-fee3-41b0-a018-4b3ccd0b38d0 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:11.392847 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10237950-fee3-41b0-a018-4b3ccd0b38d0 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:11.407980 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9e733-53b6-4599-821a-0610a030b379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:11.408947 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9e733-53b6-4599-821a-0610a030b379 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:11.409396 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9e733-53b6-4599-821a-0610a030b379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:11.409849 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9e733-53b6-4599-821a-0610a030b379 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:11.410748 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c9e733-53b6-4599-821a-0610a030b379 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:11.412609 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c9e733-53b6-4599-821a-0610a030b379 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:11.427148 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 939/1893] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:17:11 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:17:11.430965 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da30b00-6cc2-46d6-a788-6c2ff4cf9bb8 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:11.442940 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da30b00-6cc2-46d6-a788-6c2ff4cf9bb8 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:11.450089 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c7c98-b803-4276-803c-1105cb7f02ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:11.451151 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c7c98-b803-4276-803c-1105cb7f02ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:11.451646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c7c98-b803-4276-803c-1105cb7f02ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:11.451997 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c7c98-b803-4276-803c-1105cb7f02ef admin admin] PATH_INFO: `/v3/projects/ad0d5a6f3ef44bcabda6a2b47b99ee58/users/c39eec238b4a4da783d8cc8b700b688f/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:11.484292 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2c7c98-b803-4276-803c-1105cb7f02ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad0d5a6f3ef44bcabda6a2b47b99ee58, user_id=c39eec238b4a4da783d8cc8b700b688f, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:11.487112 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2c7c98-b803-4276-803c-1105cb7f02ef admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:11.511020 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 955/1894] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:17:11 2022] PUT /identity/v3/projects/ad0d5a6f3ef44bcabda6a2b47b99ee58/users/c39eec238b4a4da783d8cc8b700b688f/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:17:11.522155 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dcfc1-5e77-48e8-82e5-1b04e119cb65 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:11.522632 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dcfc1-5e77-48e8-82e5-1b04e119cb65 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:11.523011 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dcfc1-5e77-48e8-82e5-1b04e119cb65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:11.567061 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-e14dcfc1-5e77-48e8-82e5-1b04e119cb65 None None] MFA Rules not processed for user `c39eec238b4a4da783d8cc8b700b688f`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:11.568622 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e14dcfc1-5e77-48e8-82e5-1b04e119cb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:11.646265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 940/1895] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:17:11.650511 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c7c98-b803-4276-803c-1105cb7f02ef admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:11.683455 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:11.714664 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c7c98-b803-4276-803c-1105cb7f02ef admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:11.719344 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3dbd70-963d-49a4-a900-4770f5f25c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:11.719998 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3dbd70-963d-49a4-a900-4770f5f25c06 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:11.720297 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3dbd70-963d-49a4-a900-4770f5f25c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:11.720551 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3dbd70-963d-49a4-a900-4770f5f25c06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:11.721265 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3dbd70-963d-49a4-a900-4770f5f25c06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:11.723140 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3dbd70-963d-49a4-a900-4770f5f25c06 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:11.746881 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 956/1896] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:17:11 2022] POST /identity/v3/projects => generated 375 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:17:11.751752 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14dcfc1-5e77-48e8-82e5-1b04e119cb65 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:11.762503 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14dcfc1-5e77-48e8-82e5-1b04e119cb65 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:11.825585 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481dbeca-a901-43e3-b87a-48f7cb3d4433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:11.826247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481dbeca-a901-43e3-b87a-48f7cb3d4433 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:11.826533 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481dbeca-a901-43e3-b87a-48f7cb3d4433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:11.826812 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481dbeca-a901-43e3-b87a-48f7cb3d4433 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:11.827531 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481dbeca-a901-43e3-b87a-48f7cb3d4433 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:11.829429 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481dbeca-a901-43e3-b87a-48f7cb3d4433 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:11.848579 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 941/1897] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:17:11 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:17:11.852689 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3dbd70-963d-49a4-a900-4770f5f25c06 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:11.863696 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3dbd70-963d-49a4-a900-4770f5f25c06 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:11.868345 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346a7ee-3b87-41e8-a8ca-02e88bca45f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:11.869001 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346a7ee-3b87-41e8-a8ca-02e88bca45f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:11.869286 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346a7ee-3b87-41e8-a8ca-02e88bca45f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:11.869569 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346a7ee-3b87-41e8-a8ca-02e88bca45f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:11.870313 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346a7ee-3b87-41e8-a8ca-02e88bca45f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:11.871939 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346a7ee-3b87-41e8-a8ca-02e88bca45f9 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:11.882939 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 957/1898] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:17:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:17:11.886658 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481dbeca-a901-43e3-b87a-48f7cb3d4433 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:11.896564 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481dbeca-a901-43e3-b87a-48f7cb3d4433 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:11.901247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e2dad7-6511-4044-9a51-ca6c2f7cdbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:11.902395 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2dad7-6511-4044-9a51-ca6c2f7cdbec admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:11.902734 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2dad7-6511-4044-9a51-ca6c2f7cdbec admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:11.903020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2dad7-6511-4044-9a51-ca6c2f7cdbec admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:11.903801 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e2dad7-6511-4044-9a51-ca6c2f7cdbec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:11.905437 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e2dad7-6511-4044-9a51-ca6c2f7cdbec admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:11.919288 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 942/1899] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:17:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:17:11.920888 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346a7ee-3b87-41e8-a8ca-02e88bca45f9 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:11.930995 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346a7ee-3b87-41e8-a8ca-02e88bca45f9 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:11.936149 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737f71a-378d-4815-a5fd-ba17dded3166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:11.937036 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737f71a-378d-4815-a5fd-ba17dded3166 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:11.937318 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737f71a-378d-4815-a5fd-ba17dded3166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:11.937592 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737f71a-378d-4815-a5fd-ba17dded3166 admin admin] PATH_INFO: `/v3/projects/a933a460f53b482a8e13a39b05bca43c/users/9ae47c0d22014f7bb898577637b4bbac/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:11.965337 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0737f71a-378d-4815-a5fd-ba17dded3166 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a933a460f53b482a8e13a39b05bca43c, user_id=9ae47c0d22014f7bb898577637b4bbac, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:11.968399 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0737f71a-378d-4815-a5fd-ba17dded3166 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:11.990354 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 958/1900] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:17:11 2022] PUT /identity/v3/projects/a933a460f53b482a8e13a39b05bca43c/users/9ae47c0d22014f7bb898577637b4bbac/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:12.000934 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ebaec-947d-454d-9b27-9c2f11171b02 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:12.001234 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ebaec-947d-454d-9b27-9c2f11171b02 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:12.001516 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ebaec-947d-454d-9b27-9c2f11171b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:12.028999 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-e95ebaec-947d-454d-9b27-9c2f11171b02 None None] MFA Rules not processed for user `9ae47c0d22014f7bb898577637b4bbac`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:12.030231 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e95ebaec-947d-454d-9b27-9c2f11171b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:12.070517 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:12.098219 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 943/1901] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:17:12.102776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737f71a-378d-4815-a5fd-ba17dded3166 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:12.167559 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737f71a-378d-4815-a5fd-ba17dded3166 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:12.172763 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098fecb-ebe4-4b26-8d3f-ce5e0f0a6bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:12.173421 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098fecb-ebe4-4b26-8d3f-ce5e0f0a6bff admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:12.173713 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098fecb-ebe4-4b26-8d3f-ce5e0f0a6bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:12.173989 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098fecb-ebe4-4b26-8d3f-ce5e0f0a6bff admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:12.174688 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a098fecb-ebe4-4b26-8d3f-ce5e0f0a6bff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:12.176604 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a098fecb-ebe4-4b26-8d3f-ce5e0f0a6bff admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:12.200909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 959/1902] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:17:12 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:17:12.205334 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ebaec-947d-454d-9b27-9c2f11171b02 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:12.216801 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ebaec-947d-454d-9b27-9c2f11171b02 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:12.281965 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423f5fd8-7738-4ee4-b8e8-c2b170ac334b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:12.282815 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f5fd8-7738-4ee4-b8e8-c2b170ac334b admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:12.283192 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f5fd8-7738-4ee4-b8e8-c2b170ac334b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:12.283556 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f5fd8-7738-4ee4-b8e8-c2b170ac334b admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:12.284456 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423f5fd8-7738-4ee4-b8e8-c2b170ac334b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:12.286408 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423f5fd8-7738-4ee4-b8e8-c2b170ac334b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:12.308066 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 944/1903] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:17:12 2022] POST /identity/v3/users => generated 295 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:17:12.312242 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098fecb-ebe4-4b26-8d3f-ce5e0f0a6bff admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:12.328562 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098fecb-ebe4-4b26-8d3f-ce5e0f0a6bff admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:12.334112 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1756b27-dc87-494e-a54b-b43d91430f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:12.334792 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1756b27-dc87-494e-a54b-b43d91430f57 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:12.335081 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1756b27-dc87-494e-a54b-b43d91430f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:12.335520 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1756b27-dc87-494e-a54b-b43d91430f57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:12.336762 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1756b27-dc87-494e-a54b-b43d91430f57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:12.338429 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1756b27-dc87-494e-a54b-b43d91430f57 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:12.358930 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 960/1904] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:17:12 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:17:12.363466 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423f5fd8-7738-4ee4-b8e8-c2b170ac334b admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:12.375061 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423f5fd8-7738-4ee4-b8e8-c2b170ac334b admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:12.389256 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2162e23b-0d3e-4a15-a2b9-a126353f5967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:12.389256 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162e23b-0d3e-4a15-a2b9-a126353f5967 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:12.389256 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162e23b-0d3e-4a15-a2b9-a126353f5967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:12.389256 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162e23b-0d3e-4a15-a2b9-a126353f5967 admin admin] PATH_INFO: `/v3/projects/e878dea8fb424003affbc951848ff041/users/c5681e361dcb44d18cafe8c5cc0a5d73/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:12.399138 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2162e23b-0d3e-4a15-a2b9-a126353f5967 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e878dea8fb424003affbc951848ff041, user_id=c5681e361dcb44d18cafe8c5cc0a5d73, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:12.403010 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2162e23b-0d3e-4a15-a2b9-a126353f5967 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:12.426647 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 945/1905] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:17:12 2022] PUT /identity/v3/projects/e878dea8fb424003affbc951848ff041/users/c5681e361dcb44d18cafe8c5cc0a5d73/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:12.430686 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1756b27-dc87-494e-a54b-b43d91430f57 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:12.441528 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1756b27-dc87-494e-a54b-b43d91430f57 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:12.496450 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:12.505274 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ee9c2-d78e-41a5-9114-56933899e645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:12.506082 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ee9c2-d78e-41a5-9114-56933899e645 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:12.506472 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ee9c2-d78e-41a5-9114-56933899e645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:12.506852 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ee9c2-d78e-41a5-9114-56933899e645 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:12.507965 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ee9c2-d78e-41a5-9114-56933899e645 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:12.509643 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ee9c2-d78e-41a5-9114-56933899e645 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:12.522414 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 961/1906] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:17:12 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:17:12.527056 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2162e23b-0d3e-4a15-a2b9-a126353f5967 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:12.539666 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2162e23b-0d3e-4a15-a2b9-a126353f5967 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:12.544639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9137d2b1-2a16-41b5-b3e8-8dd8bd1320a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:12.545910 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9137d2b1-2a16-41b5-b3e8-8dd8bd1320a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:12.545910 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9137d2b1-2a16-41b5-b3e8-8dd8bd1320a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:12.546156 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9137d2b1-2a16-41b5-b3e8-8dd8bd1320a6 admin admin] PATH_INFO: `/v3/projects/e878dea8fb424003affbc951848ff041/users/c5681e361dcb44d18cafe8c5cc0a5d73/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:12.548124 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9137d2b1-2a16-41b5-b3e8-8dd8bd1320a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e878dea8fb424003affbc951848ff041, user_id=c5681e361dcb44d18cafe8c5cc0a5d73, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:12.550770 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9137d2b1-2a16-41b5-b3e8-8dd8bd1320a6 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:12.565873 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 946/1907] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:17:12 2022] PUT /identity/v3/projects/e878dea8fb424003affbc951848ff041/users/c5681e361dcb44d18cafe8c5cc0a5d73/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:12.578670 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49feeb00-4c84-4627-98f6-80e0602a03c2 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:12.578967 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49feeb00-4c84-4627-98f6-80e0602a03c2 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:12.579250 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49feeb00-4c84-4627-98f6-80e0602a03c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:12.613569 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-49feeb00-4c84-4627-98f6-80e0602a03c2 None None] MFA Rules not processed for user `c5681e361dcb44d18cafe8c5cc0a5d73`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:12.614805 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-49feeb00-4c84-4627-98f6-80e0602a03c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:12.757258 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 962/1908] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:12 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:17:12.773009 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9137d2b1-2a16-41b5-b3e8-8dd8bd1320a6 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:12.778865 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9137d2b1-2a16-41b5-b3e8-8dd8bd1320a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:12.839196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9137d2b1-2a16-41b5-b3e8-8dd8bd1320a6 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:12.842853 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a3696-eb74-4a51-9f03-0cd2f391fbeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:12.843527 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a3696-eb74-4a51-9f03-0cd2f391fbeb service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:12.843814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a3696-eb74-4a51-9f03-0cd2f391fbeb service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:12.844091 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a3696-eb74-4a51-9f03-0cd2f391fbeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:12.887975 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:12.900868 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a3696-eb74-4a51-9f03-0cd2f391fbeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:12.902550 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a3696-eb74-4a51-9f03-0cd2f391fbeb service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:12.911763 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 947/1909] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:17:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:17:13.082971 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49feeb00-4c84-4627-98f6-80e0602a03c2 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:13.084585 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-49feeb00-4c84-4627-98f6-80e0602a03c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:13.164568 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49feeb00-4c84-4627-98f6-80e0602a03c2 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:13.169956 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ef431-1a36-4020-bed2-40faa9c33769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:13.170852 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ef431-1a36-4020-bed2-40faa9c33769 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:13.171888 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ef431-1a36-4020-bed2-40faa9c33769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:13.171888 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ef431-1a36-4020-bed2-40faa9c33769 admin admin] PATH_INFO: `/v3/users/95d940dcc628403295c125352a2c95e4` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:13.172830 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ef431-1a36-4020-bed2-40faa9c33769 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95d940dcc628403295c125352a2c95e4)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:13.174877 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ef431-1a36-4020-bed2-40faa9c33769 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:13.239213 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-024ef431-1a36-4020-bed2-40faa9c33769 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95d940dcc628403295c125352a2c95e4'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:13.247729 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-024ef431-1a36-4020-bed2-40faa9c33769 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95d940dcc628403295c125352a2c95e4'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:13.267132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-024ef431-1a36-4020-bed2-40faa9c33769 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95d940dcc628403295c125352a2c95e4'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:13.275949 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:13.279798 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-024ef431-1a36-4020-bed2-40faa9c33769 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95d940dcc628403295c125352a2c95e4'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:13.280919 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-024ef431-1a36-4020-bed2-40faa9c33769 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95d940dcc628403295c125352a2c95e4'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:13.291351 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 963/1910] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:17:13 2022] DELETE /identity/v3/users/95d940dcc628403295c125352a2c95e4 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:13.293703 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf48bc-05c8-4e07-85f5-4e572f505750 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:13.294279 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf48bc-05c8-4e07-85f5-4e572f505750 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:13.294279 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf48bc-05c8-4e07-85f5-4e572f505750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:13.311387 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-4acf48bc-05c8-4e07-85f5-4e572f505750 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:13.312622 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4acf48bc-05c8-4e07-85f5-4e572f505750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:13.370665 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 948/1911] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:17:13.380253 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ef431-1a36-4020-bed2-40faa9c33769 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:13.381641 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-024ef431-1a36-4020-bed2-40faa9c33769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:13.440792 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ef431-1a36-4020-bed2-40faa9c33769 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:13.444755 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce2b93-0117-4626-950c-ae4bfed1990a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:13.445350 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce2b93-0117-4626-950c-ae4bfed1990a service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:13.445634 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce2b93-0117-4626-950c-ae4bfed1990a service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:13.445912 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce2b93-0117-4626-950c-ae4bfed1990a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:13.457428 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ce2b93-0117-4626-950c-ae4bfed1990a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:13.458890 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ce2b93-0117-4626-950c-ae4bfed1990a service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:13.469090 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 964/1912] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:17:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:17:13.776753 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf48bc-05c8-4e07-85f5-4e572f505750 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:13.776753 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4acf48bc-05c8-4e07-85f5-4e572f505750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:13.785820 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf48bc-05c8-4e07-85f5-4e572f505750 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:13.791355 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9745c5-c574-4362-83dd-b74264f53eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:13.792350 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9745c5-c574-4362-83dd-b74264f53eba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:13.792723 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9745c5-c574-4362-83dd-b74264f53eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:13.793302 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9745c5-c574-4362-83dd-b74264f53eba admin admin] PATH_INFO: `/v3/projects/90e7d35a6d114ab7b793b452e40904d5` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:13.794460 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9745c5-c574-4362-83dd-b74264f53eba admin admin] RBAC: Authorizing `identity:delete_project(project_id=90e7d35a6d114ab7b793b452e40904d5)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:13.796563 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9745c5-c574-4362-83dd-b74264f53eba admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:13.907305 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2a9745c5-c574-4362-83dd-b74264f53eba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90e7d35a6d114ab7b793b452e40904d5', 'request_id': 'req-2a9745c5-c574-4362-83dd-b74264f53eba'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:13.924161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2a9745c5-c574-4362-83dd-b74264f53eba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90e7d35a6d114ab7b793b452e40904d5', 'request_id': 'req-2a9745c5-c574-4362-83dd-b74264f53eba'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:13.941353 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2a9745c5-c574-4362-83dd-b74264f53eba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90e7d35a6d114ab7b793b452e40904d5', 'request_id': 'req-2a9745c5-c574-4362-83dd-b74264f53eba'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:13.962479 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2a9745c5-c574-4362-83dd-b74264f53eba admin admin] The token cache is being invalidate because project 90e7d35a6d114ab7b793b452e40904d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:17:13.962701 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2a9745c5-c574-4362-83dd-b74264f53eba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:13.964473 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2a9745c5-c574-4362-83dd-b74264f53eba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:13.971324 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 949/1913] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:17:13 2022] DELETE /identity/v3/projects/90e7d35a6d114ab7b793b452e40904d5 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:13.975359 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce2b93-0117-4626-950c-ae4bfed1990a service neutron] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:13.989730 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce2b93-0117-4626-950c-ae4bfed1990a service neutron] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:14.070654 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65267704-1e8f-4023-88c3-e37e0bae3e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:14.072663 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65267704-1e8f-4023-88c3-e37e0bae3e3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:14.074120 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65267704-1e8f-4023-88c3-e37e0bae3e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:14.074433 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65267704-1e8f-4023-88c3-e37e0bae3e3f admin admin] PATH_INFO: `/v3/users/e26549614eff43b2a69e2a2cfac9a4b6` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:14.076874 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65267704-1e8f-4023-88c3-e37e0bae3e3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e26549614eff43b2a69e2a2cfac9a4b6)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:14.079359 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65267704-1e8f-4023-88c3-e37e0bae3e3f admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:14.160423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-65267704-1e8f-4023-88c3-e37e0bae3e3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e26549614eff43b2a69e2a2cfac9a4b6'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:14.173070 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-65267704-1e8f-4023-88c3-e37e0bae3e3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e26549614eff43b2a69e2a2cfac9a4b6'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:14.188056 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-65267704-1e8f-4023-88c3-e37e0bae3e3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e26549614eff43b2a69e2a2cfac9a4b6'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:14.199852 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-65267704-1e8f-4023-88c3-e37e0bae3e3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e26549614eff43b2a69e2a2cfac9a4b6'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:14.200869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-65267704-1e8f-4023-88c3-e37e0bae3e3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e26549614eff43b2a69e2a2cfac9a4b6'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:14.210108 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 965/1914] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:17:13 2022] DELETE /identity/v3/users/e26549614eff43b2a69e2a2cfac9a4b6 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:17:14.475376 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9745c5-c574-4362-83dd-b74264f53eba admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:14.477282 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2a9745c5-c574-4362-83dd-b74264f53eba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:14.512327 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65267704-1e8f-4023-88c3-e37e0bae3e3f admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:14.514023 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-65267704-1e8f-4023-88c3-e37e0bae3e3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:14.539124 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:14.552479 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9745c5-c574-4362-83dd-b74264f53eba admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:14.573029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:14.588895 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65267704-1e8f-4023-88c3-e37e0bae3e3f admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:14.594055 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ff69b-2afa-4171-8094-3396b6b4d02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:14.594990 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ff69b-2afa-4171-8094-3396b6b4d02d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:14.595413 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ff69b-2afa-4171-8094-3396b6b4d02d admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:14.595683 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ff69b-2afa-4171-8094-3396b6b4d02d admin admin] PATH_INFO: `/v3/projects/a18b838e6b384a8a923d46960e552f2a` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:14.596853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432ff69b-2afa-4171-8094-3396b6b4d02d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a18b838e6b384a8a923d46960e552f2a)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:14.598777 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432ff69b-2afa-4171-8094-3396b6b4d02d admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:14.604789 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a755c-55ea-4990-b327-d06e3e4dab98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:14.605657 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a755c-55ea-4990-b327-d06e3e4dab98 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:14.606121 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a755c-55ea-4990-b327-d06e3e4dab98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:14.606967 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a755c-55ea-4990-b327-d06e3e4dab98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:14.608778 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d94a755c-55ea-4990-b327-d06e3e4dab98 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:14.647162 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-432ff69b-2afa-4171-8094-3396b6b4d02d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a18b838e6b384a8a923d46960e552f2a', 'request_id': 'req-432ff69b-2afa-4171-8094-3396b6b4d02d'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:14.654246 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-432ff69b-2afa-4171-8094-3396b6b4d02d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a18b838e6b384a8a923d46960e552f2a', 'request_id': 'req-432ff69b-2afa-4171-8094-3396b6b4d02d'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:14.662562 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-432ff69b-2afa-4171-8094-3396b6b4d02d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a18b838e6b384a8a923d46960e552f2a', 'request_id': 'req-432ff69b-2afa-4171-8094-3396b6b4d02d'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:14.667641 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a755c-55ea-4990-b327-d06e3e4dab98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:14.669655 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a755c-55ea-4990-b327-d06e3e4dab98 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:14.674169 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-432ff69b-2afa-4171-8094-3396b6b4d02d admin admin] The token cache is being invalidate because project a18b838e6b384a8a923d46960e552f2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:17:14.674471 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-432ff69b-2afa-4171-8094-3396b6b4d02d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:14.675128 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-432ff69b-2afa-4171-8094-3396b6b4d02d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:14.676724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 966/1915] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:17:14 2022] DELETE /identity/v3/projects/a18b838e6b384a8a923d46960e552f2a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:17:14.678901 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 950/1916] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:17:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:17:14.680658 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ff69b-2afa-4171-8094-3396b6b4d02d admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:14.681970 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-432ff69b-2afa-4171-8094-3396b6b4d02d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:14.745035 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ff69b-2afa-4171-8094-3396b6b4d02d admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:14.749882 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896285f3-3a9c-4c26-aa53-79c528f75f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:14.750752 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896285f3-3a9c-4c26-aa53-79c528f75f2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:14.751028 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896285f3-3a9c-4c26-aa53-79c528f75f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:14.751318 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896285f3-3a9c-4c26-aa53-79c528f75f2d admin admin] PATH_INFO: `/v3/users/61c162da02a64cb9aa81fe5adc51d1ff` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:14.752514 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896285f3-3a9c-4c26-aa53-79c528f75f2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=61c162da02a64cb9aa81fe5adc51d1ff)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:14.754471 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896285f3-3a9c-4c26-aa53-79c528f75f2d admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:14.818004 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-896285f3-3a9c-4c26-aa53-79c528f75f2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61c162da02a64cb9aa81fe5adc51d1ff'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:14.821842 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:14.832087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-896285f3-3a9c-4c26-aa53-79c528f75f2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61c162da02a64cb9aa81fe5adc51d1ff'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:14.851413 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-896285f3-3a9c-4c26-aa53-79c528f75f2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61c162da02a64cb9aa81fe5adc51d1ff'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:14.860409 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-896285f3-3a9c-4c26-aa53-79c528f75f2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61c162da02a64cb9aa81fe5adc51d1ff'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:14.861296 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-896285f3-3a9c-4c26-aa53-79c528f75f2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61c162da02a64cb9aa81fe5adc51d1ff'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:14.868033 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 967/1917] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:17:14 2022] DELETE /identity/v3/users/61c162da02a64cb9aa81fe5adc51d1ff => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:15.245642 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a755c-55ea-4990-b327-d06e3e4dab98 service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:15.266452 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a755c-55ea-4990-b327-d06e3e4dab98 service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:15.330908 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130016a8-20f1-45b3-a478-2f41eb77c58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:15.331812 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130016a8-20f1-45b3-a478-2f41eb77c58c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:15.332088 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130016a8-20f1-45b3-a478-2f41eb77c58c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:15.332372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130016a8-20f1-45b3-a478-2f41eb77c58c admin admin] PATH_INFO: `/v3/projects/2c7ff9568fcd4951892e77dcc0f2b9c1` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:15.333568 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130016a8-20f1-45b3-a478-2f41eb77c58c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c7ff9568fcd4951892e77dcc0f2b9c1)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:15.335542 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130016a8-20f1-45b3-a478-2f41eb77c58c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:15.389784 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-130016a8-20f1-45b3-a478-2f41eb77c58c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c7ff9568fcd4951892e77dcc0f2b9c1', 'request_id': 'req-130016a8-20f1-45b3-a478-2f41eb77c58c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:15.397382 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-130016a8-20f1-45b3-a478-2f41eb77c58c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c7ff9568fcd4951892e77dcc0f2b9c1', 'request_id': 'req-130016a8-20f1-45b3-a478-2f41eb77c58c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:15.405837 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-130016a8-20f1-45b3-a478-2f41eb77c58c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c7ff9568fcd4951892e77dcc0f2b9c1', 'request_id': 'req-130016a8-20f1-45b3-a478-2f41eb77c58c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:15.416945 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-130016a8-20f1-45b3-a478-2f41eb77c58c admin admin] The token cache is being invalidate because project 2c7ff9568fcd4951892e77dcc0f2b9c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:17:15.417683 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-130016a8-20f1-45b3-a478-2f41eb77c58c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:15.418453 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-130016a8-20f1-45b3-a478-2f41eb77c58c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:15.420838 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 951/1918] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:17:15 2022] DELETE /identity/v3/projects/2c7ff9568fcd4951892e77dcc0f2b9c1 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:15.426917 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933408d-6938-41b8-86df-b649a7cd0873 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:15.427229 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933408d-6938-41b8-86df-b649a7cd0873 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:15.427549 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933408d-6938-41b8-86df-b649a7cd0873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:15.445666 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-7933408d-6938-41b8-86df-b649a7cd0873 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:15.446876 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-7933408d-6938-41b8-86df-b649a7cd0873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:15.460074 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130016a8-20f1-45b3-a478-2f41eb77c58c admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:15.461484 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-130016a8-20f1-45b3-a478-2f41eb77c58c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:15.474276 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:15.517625 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 968/1919] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:17:15.525547 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8c71f-8952-4f6d-a6a2-79012437987f None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:15.526031 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8c71f-8952-4f6d-a6a2-79012437987f None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:15.526486 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8c71f-8952-4f6d-a6a2-79012437987f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:15.536671 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130016a8-20f1-45b3-a478-2f41eb77c58c admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:15.540481 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696cd44-6f50-4496-b4a6-ba6afc4fec6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:15.541131 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696cd44-6f50-4496-b4a6-ba6afc4fec6d service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:15.541415 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696cd44-6f50-4496-b4a6-ba6afc4fec6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:15.542086 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696cd44-6f50-4496-b4a6-ba6afc4fec6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:15.543546 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8696cd44-6f50-4496-b4a6-ba6afc4fec6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:15.548505 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-a4d8c71f-8952-4f6d-a6a2-79012437987f None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:15.550194 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-a4d8c71f-8952-4f6d-a6a2-79012437987f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:15.559740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 969/1920] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:17:15.563879 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d8c71f-8952-4f6d-a6a2-79012437987f None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:15.573752 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d8c71f-8952-4f6d-a6a2-79012437987f None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:15.579098 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ff19ba-2664-46d0-8192-562b14646434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:15.579832 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff19ba-2664-46d0-8192-562b14646434 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:15.580112 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff19ba-2664-46d0-8192-562b14646434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:15.580388 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff19ba-2664-46d0-8192-562b14646434 admin admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:15.581111 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ff19ba-2664-46d0-8192-562b14646434 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:17:15.581418 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ff19ba-2664-46d0-8192-562b14646434 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:15.583023 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ff19ba-2664-46d0-8192-562b14646434 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:15.596162 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 970/1921] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:17:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:17:15.601051 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ff19ba-2664-46d0-8192-562b14646434 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:15.610699 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ff19ba-2664-46d0-8192-562b14646434 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:15.615435 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12676aba-8a9c-4c15-8733-8184a57ae65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:15.616084 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12676aba-8a9c-4c15-8733-8184a57ae65a admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:15.616368 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12676aba-8a9c-4c15-8733-8184a57ae65a admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:15.616640 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12676aba-8a9c-4c15-8733-8184a57ae65a admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:15.617321 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12676aba-8a9c-4c15-8733-8184a57ae65a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:15.619207 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12676aba-8a9c-4c15-8733-8184a57ae65a admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:15.624576 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8696cd44-6f50-4496-b4a6-ba6afc4fec6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:15.626021 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8696cd44-6f50-4496-b4a6-ba6afc4fec6d service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:15.640128 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 952/1922] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:17:15 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:17:15.646778 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 971/1923] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:17:15 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:17:15.651495 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696cd44-6f50-4496-b4a6-ba6afc4fec6d service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:15.668461 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696cd44-6f50-4496-b4a6-ba6afc4fec6d service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:15.728301 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bda5a3-5168-4b10-a0df-f25c8bf9ef6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:15.729003 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bda5a3-5168-4b10-a0df-f25c8bf9ef6f admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:15.729291 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bda5a3-5168-4b10-a0df-f25c8bf9ef6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:15.729569 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bda5a3-5168-4b10-a0df-f25c8bf9ef6f admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:15.730265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bda5a3-5168-4b10-a0df-f25c8bf9ef6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:15.732247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bda5a3-5168-4b10-a0df-f25c8bf9ef6f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:15.752484 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 953/1924] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:17:15 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:17:15.756515 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12676aba-8a9c-4c15-8733-8184a57ae65a admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:15.767723 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12676aba-8a9c-4c15-8733-8184a57ae65a admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:15.772228 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567a495-64d1-41e7-8319-125f8e2c1807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:15.772878 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567a495-64d1-41e7-8319-125f8e2c1807 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:15.773160 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567a495-64d1-41e7-8319-125f8e2c1807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:15.773438 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567a495-64d1-41e7-8319-125f8e2c1807 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:15.774152 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1567a495-64d1-41e7-8319-125f8e2c1807 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:15.775754 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1567a495-64d1-41e7-8319-125f8e2c1807 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:15.786300 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 972/1925] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:17:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:17:15.790154 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bda5a3-5168-4b10-a0df-f25c8bf9ef6f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:15.799972 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bda5a3-5168-4b10-a0df-f25c8bf9ef6f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:15.804655 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2f460d-c2d0-4c13-9d7b-d1f68163aab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:15.805312 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f460d-c2d0-4c13-9d7b-d1f68163aab9 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:15.805597 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f460d-c2d0-4c13-9d7b-d1f68163aab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:15.805873 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f460d-c2d0-4c13-9d7b-d1f68163aab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:15.806605 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2f460d-c2d0-4c13-9d7b-d1f68163aab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:15.808245 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2f460d-c2d0-4c13-9d7b-d1f68163aab9 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:15.817921 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 954/1926] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:17:15 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:17:15.821830 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567a495-64d1-41e7-8319-125f8e2c1807 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:15.835437 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567a495-64d1-41e7-8319-125f8e2c1807 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:15.843631 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce71ae-4992-49a0-8e1f-5b477c98dc36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:15.845308 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce71ae-4992-49a0-8e1f-5b477c98dc36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:15.845825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce71ae-4992-49a0-8e1f-5b477c98dc36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:15.846337 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce71ae-4992-49a0-8e1f-5b477c98dc36 admin admin] PATH_INFO: `/v3/projects/c3d852874a8f40c2a7aa1e9e49f98748/users/e06302d5f48c4e609f2ba3033a75ca8e/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:15.868409 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ce71ae-4992-49a0-8e1f-5b477c98dc36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3d852874a8f40c2a7aa1e9e49f98748, user_id=e06302d5f48c4e609f2ba3033a75ca8e, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:15.871048 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ce71ae-4992-49a0-8e1f-5b477c98dc36 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:15.884378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 973/1927] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:17:15 2022] PUT /identity/v3/projects/c3d852874a8f40c2a7aa1e9e49f98748/users/e06302d5f48c4e609f2ba3033a75ca8e/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:15.897322 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537e24d-e0b9-497e-abbd-2efa3af1e625 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:15.897614 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537e24d-e0b9-497e-abbd-2efa3af1e625 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:15.897895 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537e24d-e0b9-497e-abbd-2efa3af1e625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:15.923867 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c537e24d-e0b9-497e-abbd-2efa3af1e625 None None] MFA Rules not processed for user `e06302d5f48c4e609f2ba3033a75ca8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:15.925062 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c537e24d-e0b9-497e-abbd-2efa3af1e625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:15.954322 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:15.994327 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 955/1928] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:17:15.999462 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce71ae-4992-49a0-8e1f-5b477c98dc36 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:16.050308 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:16.063756 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce71ae-4992-49a0-8e1f-5b477c98dc36 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:16.069666 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fea1d0-859a-4674-a647-5399cc6eb027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:16.070329 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fea1d0-859a-4674-a647-5399cc6eb027 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:16.070612 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fea1d0-859a-4674-a647-5399cc6eb027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:16.070884 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fea1d0-859a-4674-a647-5399cc6eb027 admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:16.071608 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fea1d0-859a-4674-a647-5399cc6eb027 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:16.073502 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fea1d0-859a-4674-a647-5399cc6eb027 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:16.103153 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 974/1929] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:17:15 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:17:16.104320 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537e24d-e0b9-497e-abbd-2efa3af1e625 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:16.114888 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537e24d-e0b9-497e-abbd-2efa3af1e625 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:16.170524 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58e7884-80cf-4aff-82b3-36147dd8986f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:16.171198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e7884-80cf-4aff-82b3-36147dd8986f admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:16.171545 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e7884-80cf-4aff-82b3-36147dd8986f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:16.171871 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e7884-80cf-4aff-82b3-36147dd8986f admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:16.172572 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58e7884-80cf-4aff-82b3-36147dd8986f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:16.174521 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58e7884-80cf-4aff-82b3-36147dd8986f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:16.193958 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 956/1930] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:17:16 2022] POST /identity/v3/users => generated 300 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:17:16.197977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fea1d0-859a-4674-a647-5399cc6eb027 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:16.208693 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fea1d0-859a-4674-a647-5399cc6eb027 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:16.213283 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9eb1a7-2421-4d05-a3e1-2a2ad3ae1205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:16.213964 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9eb1a7-2421-4d05-a3e1-2a2ad3ae1205 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:16.214249 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9eb1a7-2421-4d05-a3e1-2a2ad3ae1205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:16.214525 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9eb1a7-2421-4d05-a3e1-2a2ad3ae1205 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:16.215256 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9eb1a7-2421-4d05-a3e1-2a2ad3ae1205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:16.216851 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9eb1a7-2421-4d05-a3e1-2a2ad3ae1205 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:16.227160 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 975/1931] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:17:16.233242 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58e7884-80cf-4aff-82b3-36147dd8986f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:16.244037 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58e7884-80cf-4aff-82b3-36147dd8986f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:16.254227 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ec472-0ce6-4090-abe3-0db821c11bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:16.255163 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ec472-0ce6-4090-abe3-0db821c11bc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:16.255479 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ec472-0ce6-4090-abe3-0db821c11bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:16.255759 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ec472-0ce6-4090-abe3-0db821c11bc4 admin admin] PATH_INFO: `/v3/projects/105e5ba911c54503b5abe4e8738c9d98/users/4af36e2cf17f4e22ba52f8bb2eea6770/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:16.273500 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433ec472-0ce6-4090-abe3-0db821c11bc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=105e5ba911c54503b5abe4e8738c9d98, user_id=4af36e2cf17f4e22ba52f8bb2eea6770, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:16.276230 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433ec472-0ce6-4090-abe3-0db821c11bc4 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:16.291628 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 957/1932] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:17:16 2022] PUT /identity/v3/projects/105e5ba911c54503b5abe4e8738c9d98/users/4af36e2cf17f4e22ba52f8bb2eea6770/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:16.296177 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9eb1a7-2421-4d05-a3e1-2a2ad3ae1205 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:16.307012 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9eb1a7-2421-4d05-a3e1-2a2ad3ae1205 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:16.374847 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ac79ae-f396-434e-b2e8-038422db379c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:16.375677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac79ae-f396-434e-b2e8-038422db379c admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:16.375971 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac79ae-f396-434e-b2e8-038422db379c admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:16.376301 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac79ae-f396-434e-b2e8-038422db379c admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:16.377039 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ac79ae-f396-434e-b2e8-038422db379c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:16.378704 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ac79ae-f396-434e-b2e8-038422db379c admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:16.390033 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 976/1933] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:17:16.393848 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ec472-0ce6-4090-abe3-0db821c11bc4 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:16.404397 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ec472-0ce6-4090-abe3-0db821c11bc4 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:16.408995 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359af4c2-321d-461d-9d28-7dee671071da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:16.409882 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359af4c2-321d-461d-9d28-7dee671071da admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:16.410195 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359af4c2-321d-461d-9d28-7dee671071da admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:16.410476 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359af4c2-321d-461d-9d28-7dee671071da admin admin] PATH_INFO: `/v3/projects/105e5ba911c54503b5abe4e8738c9d98/users/4af36e2cf17f4e22ba52f8bb2eea6770/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:16.412472 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359af4c2-321d-461d-9d28-7dee671071da admin admin] RBAC: Authorizing `identity:create_grant(project_id=105e5ba911c54503b5abe4e8738c9d98, user_id=4af36e2cf17f4e22ba52f8bb2eea6770, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:16.415093 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359af4c2-321d-461d-9d28-7dee671071da admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:16.425848 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 958/1934] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:17:16 2022] PUT /identity/v3/projects/105e5ba911c54503b5abe4e8738c9d98/users/4af36e2cf17f4e22ba52f8bb2eea6770/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:16.436483 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d0360-a12c-4fb5-aa4e-04297dca95e7 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:16.436776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d0360-a12c-4fb5-aa4e-04297dca95e7 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:16.437056 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d0360-a12c-4fb5-aa4e-04297dca95e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:16.462448 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-907d0360-a12c-4fb5-aa4e-04297dca95e7 None None] MFA Rules not processed for user `4af36e2cf17f4e22ba52f8bb2eea6770`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:16.463683 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-907d0360-a12c-4fb5-aa4e-04297dca95e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:16.534430 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 977/1935] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:16 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:17:16.546257 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359af4c2-321d-461d-9d28-7dee671071da admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:16.602156 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359af4c2-321d-461d-9d28-7dee671071da admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:16.605830 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254acc31-9852-4ed9-9f9f-e566b9628d56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:16.606498 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254acc31-9852-4ed9-9f9f-e566b9628d56 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:16.606784 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254acc31-9852-4ed9-9f9f-e566b9628d56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:16.607058 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254acc31-9852-4ed9-9f9f-e566b9628d56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:16.615979 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254acc31-9852-4ed9-9f9f-e566b9628d56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:16.617402 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254acc31-9852-4ed9-9f9f-e566b9628d56 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:16.627237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 959/1936] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:17:16 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:17:17.352270 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d0360-a12c-4fb5-aa4e-04297dca95e7 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:17.400950 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:17.419717 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d0360-a12c-4fb5-aa4e-04297dca95e7 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:17.424248 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba14bbe-d820-4e96-9166-03dec5e62600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:17.425121 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba14bbe-d820-4e96-9166-03dec5e62600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:17.425402 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba14bbe-d820-4e96-9166-03dec5e62600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:17.425675 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba14bbe-d820-4e96-9166-03dec5e62600 admin admin] PATH_INFO: `/v3/users/e06302d5f48c4e609f2ba3033a75ca8e` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:17.426854 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba14bbe-d820-4e96-9166-03dec5e62600 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e06302d5f48c4e609f2ba3033a75ca8e)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:17.428790 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba14bbe-d820-4e96-9166-03dec5e62600 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:17.482089 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-aba14bbe-d820-4e96-9166-03dec5e62600 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e06302d5f48c4e609f2ba3033a75ca8e'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:17.491972 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-aba14bbe-d820-4e96-9166-03dec5e62600 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e06302d5f48c4e609f2ba3033a75ca8e'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:17.507189 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-aba14bbe-d820-4e96-9166-03dec5e62600 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e06302d5f48c4e609f2ba3033a75ca8e'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:17.515398 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-aba14bbe-d820-4e96-9166-03dec5e62600 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e06302d5f48c4e609f2ba3033a75ca8e'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:17.516256 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-aba14bbe-d820-4e96-9166-03dec5e62600 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e06302d5f48c4e609f2ba3033a75ca8e'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:17.525996 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 978/1937] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:17:17 2022] DELETE /identity/v3/users/e06302d5f48c4e609f2ba3033a75ca8e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:17.530978 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75887c8-889a-49a0-8151-51cf0b8919aa None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:17.531549 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75887c8-889a-49a0-8151-51cf0b8919aa None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:17.531928 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75887c8-889a-49a0-8151-51cf0b8919aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:17.554154 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b75887c8-889a-49a0-8151-51cf0b8919aa None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:17.555706 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b75887c8-889a-49a0-8151-51cf0b8919aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:17.575617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:17.619611 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 960/1938] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:17:17.629039 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba14bbe-d820-4e96-9166-03dec5e62600 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:17.630500 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-aba14bbe-d820-4e96-9166-03dec5e62600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:17.686584 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba14bbe-d820-4e96-9166-03dec5e62600 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:17.690132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ae335-df97-44cc-8597-87f6e0db9f21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:17.690770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ae335-df97-44cc-8597-87f6e0db9f21 service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:17.691053 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ae335-df97-44cc-8597-87f6e0db9f21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:17.691355 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ae335-df97-44cc-8597-87f6e0db9f21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:17.700239 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ae335-df97-44cc-8597-87f6e0db9f21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:17.701646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ae335-df97-44cc-8597-87f6e0db9f21 service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:17.710915 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 979/1939] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:17:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:17:18.062193 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75887c8-889a-49a0-8151-51cf0b8919aa None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:18.063695 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b75887c8-889a-49a0-8151-51cf0b8919aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:18.080375 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75887c8-889a-49a0-8151-51cf0b8919aa None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:18.084944 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213f643-23df-4ffa-89d0-a327811b100c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:18.085817 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213f643-23df-4ffa-89d0-a327811b100c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:18.086094 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213f643-23df-4ffa-89d0-a327811b100c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:18.086367 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213f643-23df-4ffa-89d0-a327811b100c admin admin] PATH_INFO: `/v3/projects/c3d852874a8f40c2a7aa1e9e49f98748` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:18.087592 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4213f643-23df-4ffa-89d0-a327811b100c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3d852874a8f40c2a7aa1e9e49f98748)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:18.089564 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4213f643-23df-4ffa-89d0-a327811b100c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:18.138266 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4213f643-23df-4ffa-89d0-a327811b100c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3d852874a8f40c2a7aa1e9e49f98748', 'request_id': 'req-4213f643-23df-4ffa-89d0-a327811b100c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:18.144365 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4213f643-23df-4ffa-89d0-a327811b100c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3d852874a8f40c2a7aa1e9e49f98748', 'request_id': 'req-4213f643-23df-4ffa-89d0-a327811b100c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:18.151528 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4213f643-23df-4ffa-89d0-a327811b100c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3d852874a8f40c2a7aa1e9e49f98748', 'request_id': 'req-4213f643-23df-4ffa-89d0-a327811b100c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:18.162124 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4213f643-23df-4ffa-89d0-a327811b100c admin admin] The token cache is being invalidate because project c3d852874a8f40c2a7aa1e9e49f98748 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:17:18.162409 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4213f643-23df-4ffa-89d0-a327811b100c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:18.163022 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4213f643-23df-4ffa-89d0-a327811b100c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:18.164552 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 961/1940] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:17:18 2022] DELETE /identity/v3/projects/c3d852874a8f40c2a7aa1e9e49f98748 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:18.168515 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ae335-df97-44cc-8597-87f6e0db9f21 service neutron] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:18.177889 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ae335-df97-44cc-8597-87f6e0db9f21 service neutron] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:18.232943 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e872f96-594a-4c84-a0ab-d7ef215e1362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:18.233749 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e872f96-594a-4c84-a0ab-d7ef215e1362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:18.234036 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e872f96-594a-4c84-a0ab-d7ef215e1362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:18.234318 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e872f96-594a-4c84-a0ab-d7ef215e1362 admin admin] PATH_INFO: `/v3/users/4af36e2cf17f4e22ba52f8bb2eea6770` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:18.235536 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e872f96-594a-4c84-a0ab-d7ef215e1362 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4af36e2cf17f4e22ba52f8bb2eea6770)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:18.237449 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e872f96-594a-4c84-a0ab-d7ef215e1362 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:18.247971 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:18.285764 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-2e872f96-594a-4c84-a0ab-d7ef215e1362 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4af36e2cf17f4e22ba52f8bb2eea6770'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:18.292572 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-2e872f96-594a-4c84-a0ab-d7ef215e1362 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4af36e2cf17f4e22ba52f8bb2eea6770'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:18.304202 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-2e872f96-594a-4c84-a0ab-d7ef215e1362 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4af36e2cf17f4e22ba52f8bb2eea6770'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:18.311494 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-2e872f96-594a-4c84-a0ab-d7ef215e1362 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4af36e2cf17f4e22ba52f8bb2eea6770'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:18.312141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-2e872f96-594a-4c84-a0ab-d7ef215e1362 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4af36e2cf17f4e22ba52f8bb2eea6770'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:18.318136 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 980/1941] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:17:18 2022] DELETE /identity/v3/users/4af36e2cf17f4e22ba52f8bb2eea6770 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:18.587374 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213f643-23df-4ffa-89d0-a327811b100c admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:18.588970 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4213f643-23df-4ffa-89d0-a327811b100c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:18.602037 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213f643-23df-4ffa-89d0-a327811b100c admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:18.664029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602298ad-9c45-4cb7-95fe-bb150020bb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:18.665908 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602298ad-9c45-4cb7-95fe-bb150020bb77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:18.666277 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602298ad-9c45-4cb7-95fe-bb150020bb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:18.666640 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602298ad-9c45-4cb7-95fe-bb150020bb77 admin admin] PATH_INFO: `/v3/projects/105e5ba911c54503b5abe4e8738c9d98` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:18.669289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602298ad-9c45-4cb7-95fe-bb150020bb77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=105e5ba911c54503b5abe4e8738c9d98)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:18.675382 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602298ad-9c45-4cb7-95fe-bb150020bb77 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:18.729575 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:18.732307 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-602298ad-9c45-4cb7-95fe-bb150020bb77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '105e5ba911c54503b5abe4e8738c9d98', 'request_id': 'req-602298ad-9c45-4cb7-95fe-bb150020bb77'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:18.741021 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-602298ad-9c45-4cb7-95fe-bb150020bb77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '105e5ba911c54503b5abe4e8738c9d98', 'request_id': 'req-602298ad-9c45-4cb7-95fe-bb150020bb77'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:18.750783 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-602298ad-9c45-4cb7-95fe-bb150020bb77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '105e5ba911c54503b5abe4e8738c9d98', 'request_id': 'req-602298ad-9c45-4cb7-95fe-bb150020bb77'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:18.771786 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-602298ad-9c45-4cb7-95fe-bb150020bb77 admin admin] The token cache is being invalidate because project 105e5ba911c54503b5abe4e8738c9d98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:17:18.772003 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-602298ad-9c45-4cb7-95fe-bb150020bb77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:18.772687 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-602298ad-9c45-4cb7-95fe-bb150020bb77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:18.778228 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 962/1942] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:17:18 2022] DELETE /identity/v3/projects/105e5ba911c54503b5abe4e8738c9d98 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:18.780199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2307903-81a3-40a9-a698-906e8eecbe28 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:18.781332 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2307903-81a3-40a9-a698-906e8eecbe28 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:18.781792 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2307903-81a3-40a9-a698-906e8eecbe28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:18.801859 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-a2307903-81a3-40a9-a698-906e8eecbe28 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:18.803069 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-a2307903-81a3-40a9-a698-906e8eecbe28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:18.862690 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 981/1943] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:17:18.870551 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef7621-711e-4cd7-95ce-c9f48ca53541 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:18.870887 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef7621-711e-4cd7-95ce-c9f48ca53541 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:18.871174 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef7621-711e-4cd7-95ce-c9f48ca53541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:18.889922 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-ebef7621-711e-4cd7-95ce-c9f48ca53541 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:18.891127 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ebef7621-711e-4cd7-95ce-c9f48ca53541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:18.899614 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 963/1944] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:17:18.904382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2307903-81a3-40a9-a698-906e8eecbe28 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:18.913877 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2307903-81a3-40a9-a698-906e8eecbe28 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:18.918393 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba39d92-8737-42db-88e4-c5528f1c3c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:18.919660 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba39d92-8737-42db-88e4-c5528f1c3c43 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:18.919942 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba39d92-8737-42db-88e4-c5528f1c3c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:18.920213 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba39d92-8737-42db-88e4-c5528f1c3c43 admin admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:18.920932 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba39d92-8737-42db-88e4-c5528f1c3c43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:17:18.921236 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba39d92-8737-42db-88e4-c5528f1c3c43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:18.922831 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba39d92-8737-42db-88e4-c5528f1c3c43 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:18.934020 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 982/1945] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:17:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:17:18.938229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef7621-711e-4cd7-95ce-c9f48ca53541 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:18.947344 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef7621-711e-4cd7-95ce-c9f48ca53541 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:18.952260 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87342a54-a97e-4144-8f83-b60fa3b40068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:18.952902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87342a54-a97e-4144-8f83-b60fa3b40068 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:18.953193 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87342a54-a97e-4144-8f83-b60fa3b40068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:18.953477 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87342a54-a97e-4144-8f83-b60fa3b40068 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:18.954203 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87342a54-a97e-4144-8f83-b60fa3b40068 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:18.956146 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87342a54-a97e-4144-8f83-b60fa3b40068 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:18.982002 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 964/1946] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:17:18 2022] POST /identity/v3/projects => generated 359 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:17:18.986106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba39d92-8737-42db-88e4-c5528f1c3c43 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:18.995849 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba39d92-8737-42db-88e4-c5528f1c3c43 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:19.054312 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b1219-60f8-4d93-bf21-84a3c5b7ca4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:19.054975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b1219-60f8-4d93-bf21-84a3c5b7ca4e admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:19.055269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b1219-60f8-4d93-bf21-84a3c5b7ca4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:19.055559 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b1219-60f8-4d93-bf21-84a3c5b7ca4e admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:19.056258 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13b1219-60f8-4d93-bf21-84a3c5b7ca4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:19.058255 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13b1219-60f8-4d93-bf21-84a3c5b7ca4e admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:19.081769 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 983/1947] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:17:18 2022] POST /identity/v3/users => generated 280 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:17:19.086055 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87342a54-a97e-4144-8f83-b60fa3b40068 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:19.096775 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87342a54-a97e-4144-8f83-b60fa3b40068 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:19.101376 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290b510e-6499-468c-b86b-6f3298d21e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:19.102030 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b510e-6499-468c-b86b-6f3298d21e23 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:19.102315 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b510e-6499-468c-b86b-6f3298d21e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:19.102589 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b510e-6499-468c-b86b-6f3298d21e23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:19.103314 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290b510e-6499-468c-b86b-6f3298d21e23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:19.104925 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290b510e-6499-468c-b86b-6f3298d21e23 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:19.116171 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 965/1948] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:17:19.120221 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b1219-60f8-4d93-bf21-84a3c5b7ca4e admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:19.127660 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:19.130972 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b1219-60f8-4d93-bf21-84a3c5b7ca4e admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:19.136741 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a14b7e-ac20-4fc8-87c5-76992cd641a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:19.137842 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a14b7e-ac20-4fc8-87c5-76992cd641a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:19.138363 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a14b7e-ac20-4fc8-87c5-76992cd641a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:19.138631 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a14b7e-ac20-4fc8-87c5-76992cd641a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:19.139564 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a14b7e-ac20-4fc8-87c5-76992cd641a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:19.141553 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a14b7e-ac20-4fc8-87c5-76992cd641a7 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:19.152381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 984/1949] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:17:19.157175 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290b510e-6499-468c-b86b-6f3298d21e23 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:19.167724 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290b510e-6499-468c-b86b-6f3298d21e23 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:19.172998 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c320532-9059-4361-a70b-42511dff0a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:19.173882 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c320532-9059-4361-a70b-42511dff0a22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:19.174170 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c320532-9059-4361-a70b-42511dff0a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:19.174450 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c320532-9059-4361-a70b-42511dff0a22 admin admin] PATH_INFO: `/v3/projects/4e516dfd1d5442e8b557a5fbd20c01ae/users/b22af7714a5a4ed7b061b972cd282edc/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:19.190532 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c320532-9059-4361-a70b-42511dff0a22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e516dfd1d5442e8b557a5fbd20c01ae, user_id=b22af7714a5a4ed7b061b972cd282edc, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:19.193242 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c320532-9059-4361-a70b-42511dff0a22 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:19.207216 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 966/1950] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:17:19 2022] PUT /identity/v3/projects/4e516dfd1d5442e8b557a5fbd20c01ae/users/b22af7714a5a4ed7b061b972cd282edc/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:19.218517 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856223d-5f14-4690-8a70-767d847abe2b None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:19.218810 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856223d-5f14-4690-8a70-767d847abe2b None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:19.219090 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856223d-5f14-4690-8a70-767d847abe2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:19.247929 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-0856223d-5f14-4690-8a70-767d847abe2b None None] MFA Rules not processed for user `b22af7714a5a4ed7b061b972cd282edc`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:19.249240 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-0856223d-5f14-4690-8a70-767d847abe2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:19.320984 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 985/1951] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:19 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:17:19.351154 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c320532-9059-4361-a70b-42511dff0a22 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:19.352896 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0c320532-9059-4361-a70b-42511dff0a22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:19.407912 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c320532-9059-4361-a70b-42511dff0a22 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:19.411472 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a98914-227c-422c-93ea-eca3ce31e866 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:19.412108 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a98914-227c-422c-93ea-eca3ce31e866 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:19.412395 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a98914-227c-422c-93ea-eca3ce31e866 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:19.412674 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a98914-227c-422c-93ea-eca3ce31e866 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:19.421331 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a98914-227c-422c-93ea-eca3ce31e866 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:19.422800 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a98914-227c-422c-93ea-eca3ce31e866 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:19.431381 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 967/1952] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:17:19 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:17:19.445975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0856223d-5f14-4690-8a70-767d847abe2b None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:19.527851 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0856223d-5f14-4690-8a70-767d847abe2b None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:19.532723 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2798158-7e7a-4f91-b490-054bb5f89fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:19.533738 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2798158-7e7a-4f91-b490-054bb5f89fab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:19.534021 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2798158-7e7a-4f91-b490-054bb5f89fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:19.534296 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2798158-7e7a-4f91-b490-054bb5f89fab admin admin] PATH_INFO: `/v3/users/b22af7714a5a4ed7b061b972cd282edc` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:19.535535 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2798158-7e7a-4f91-b490-054bb5f89fab admin admin] RBAC: Authorizing `identity:delete_user(user_id=b22af7714a5a4ed7b061b972cd282edc)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:19.537467 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2798158-7e7a-4f91-b490-054bb5f89fab admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:19.585934 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-e2798158-7e7a-4f91-b490-054bb5f89fab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b22af7714a5a4ed7b061b972cd282edc'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:19.593553 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-e2798158-7e7a-4f91-b490-054bb5f89fab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b22af7714a5a4ed7b061b972cd282edc'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:19.607065 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-e2798158-7e7a-4f91-b490-054bb5f89fab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b22af7714a5a4ed7b061b972cd282edc'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:19.614451 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-e2798158-7e7a-4f91-b490-054bb5f89fab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b22af7714a5a4ed7b061b972cd282edc'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:19.615323 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-e2798158-7e7a-4f91-b490-054bb5f89fab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b22af7714a5a4ed7b061b972cd282edc'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:19.621399 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 986/1953] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:17:19 2022] DELETE /identity/v3/users/b22af7714a5a4ed7b061b972cd282edc => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:19.626552 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1158f-9640-42e8-9305-b16536f48fe8 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:19.627062 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1158f-9640-42e8-9305-b16536f48fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:19.627152 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1158f-9640-42e8-9305-b16536f48fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:19.644103 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-99f1158f-9640-42e8-9305-b16536f48fe8 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:19.645447 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-99f1158f-9640-42e8-9305-b16536f48fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:19.702721 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 968/1954] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:17:19.712413 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2798158-7e7a-4f91-b490-054bb5f89fab admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:19.713811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-e2798158-7e7a-4f91-b490-054bb5f89fab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:19.719669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:19.771665 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2798158-7e7a-4f91-b490-054bb5f89fab admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:19.782812 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40368a57-241f-4301-8f73-7b887b151def service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:19.783516 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40368a57-241f-4301-8f73-7b887b151def service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:19.783809 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40368a57-241f-4301-8f73-7b887b151def service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:19.784090 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40368a57-241f-4301-8f73-7b887b151def service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:19.794461 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40368a57-241f-4301-8f73-7b887b151def service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:19.795937 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40368a57-241f-4301-8f73-7b887b151def service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:19.806049 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 987/1955] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:17:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:17:20.207702 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f1158f-9640-42e8-9305-b16536f48fe8 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:20.209223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-99f1158f-9640-42e8-9305-b16536f48fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:20.231215 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f1158f-9640-42e8-9305-b16536f48fe8 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:20.236277 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067676d-9a18-4787-9361-4a7cd57a6830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:20.237294 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067676d-9a18-4787-9361-4a7cd57a6830 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:20.237661 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067676d-9a18-4787-9361-4a7cd57a6830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:20.238004 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067676d-9a18-4787-9361-4a7cd57a6830 admin admin] PATH_INFO: `/v3/projects/4e516dfd1d5442e8b557a5fbd20c01ae` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:20.239573 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1067676d-9a18-4787-9361-4a7cd57a6830 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e516dfd1d5442e8b557a5fbd20c01ae)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:20.241690 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1067676d-9a18-4787-9361-4a7cd57a6830 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:20.260036 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:20.308699 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1067676d-9a18-4787-9361-4a7cd57a6830 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e516dfd1d5442e8b557a5fbd20c01ae', 'request_id': 'req-1067676d-9a18-4787-9361-4a7cd57a6830'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:20.315102 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1067676d-9a18-4787-9361-4a7cd57a6830 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e516dfd1d5442e8b557a5fbd20c01ae', 'request_id': 'req-1067676d-9a18-4787-9361-4a7cd57a6830'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:20.323232 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1067676d-9a18-4787-9361-4a7cd57a6830 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e516dfd1d5442e8b557a5fbd20c01ae', 'request_id': 'req-1067676d-9a18-4787-9361-4a7cd57a6830'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:20.334425 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1067676d-9a18-4787-9361-4a7cd57a6830 admin admin] The token cache is being invalidate because project 4e516dfd1d5442e8b557a5fbd20c01ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:17:20.334765 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1067676d-9a18-4787-9361-4a7cd57a6830 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:20.335461 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1067676d-9a18-4787-9361-4a7cd57a6830 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:20.337122 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 969/1956] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:17:20 2022] DELETE /identity/v3/projects/4e516dfd1d5442e8b557a5fbd20c01ae => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:20.342838 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459533f-737e-45c1-8779-0269477b4b9d None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:20.343133 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459533f-737e-45c1-8779-0269477b4b9d None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:20.343453 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459533f-737e-45c1-8779-0269477b4b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:20.361391 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-1459533f-737e-45c1-8779-0269477b4b9d None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:20.362512 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-1459533f-737e-45c1-8779-0269477b4b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:20.425933 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 988/1957] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:17:20.433543 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2af545-a160-4c76-9a3d-d0ff81c0e438 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:20.433943 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2af545-a160-4c76-9a3d-d0ff81c0e438 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:20.434737 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2af545-a160-4c76-9a3d-d0ff81c0e438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:20.452959 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-3e2af545-a160-4c76-9a3d-d0ff81c0e438 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:20.454146 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3e2af545-a160-4c76-9a3d-d0ff81c0e438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:20.461842 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 970/1958] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:17:20.466566 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1459533f-737e-45c1-8779-0269477b4b9d None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:20.475926 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1459533f-737e-45c1-8779-0269477b4b9d None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:20.480634 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7aa2bf-f094-403e-8939-2bc2bba03cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:20.481904 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7aa2bf-f094-403e-8939-2bc2bba03cd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:20.482182 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7aa2bf-f094-403e-8939-2bc2bba03cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:20.482452 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7aa2bf-f094-403e-8939-2bc2bba03cd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:20.483184 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7aa2bf-f094-403e-8939-2bc2bba03cd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:17:20.483506 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7aa2bf-f094-403e-8939-2bc2bba03cd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:20.485080 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7aa2bf-f094-403e-8939-2bc2bba03cd5 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:20.496360 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 989/1959] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:17:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:17:20.500995 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2af545-a160-4c76-9a3d-d0ff81c0e438 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:20.509920 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2af545-a160-4c76-9a3d-d0ff81c0e438 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:20.515120 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6952ac-b261-4037-9c94-e2c2e1e8ff20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:20.515788 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6952ac-b261-4037-9c94-e2c2e1e8ff20 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:20.516071 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6952ac-b261-4037-9c94-e2c2e1e8ff20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:20.516353 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6952ac-b261-4037-9c94-e2c2e1e8ff20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:20.517279 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6952ac-b261-4037-9c94-e2c2e1e8ff20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:20.519173 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6952ac-b261-4037-9c94-e2c2e1e8ff20 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:20.546970 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 971/1960] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:17:20 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:17:20.551505 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7aa2bf-f094-403e-8939-2bc2bba03cd5 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:20.562926 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7aa2bf-f094-403e-8939-2bc2bba03cd5 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:20.621290 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b6e188-53e9-4299-b9e9-08ebf4d350d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:20.622024 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6e188-53e9-4299-b9e9-08ebf4d350d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:20.622310 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6e188-53e9-4299-b9e9-08ebf4d350d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:20.622588 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6e188-53e9-4299-b9e9-08ebf4d350d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:20.623381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b6e188-53e9-4299-b9e9-08ebf4d350d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:20.625259 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b6e188-53e9-4299-b9e9-08ebf4d350d1 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:20.645708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 990/1961] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:17:20 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:17:20.650053 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6952ac-b261-4037-9c94-e2c2e1e8ff20 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:20.661657 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6952ac-b261-4037-9c94-e2c2e1e8ff20 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:20.667361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a0c711-f0ce-4ce3-842a-1b6249233ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:20.668166 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0c711-f0ce-4ce3-842a-1b6249233ae9 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:20.669196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0c711-f0ce-4ce3-842a-1b6249233ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:20.669478 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0c711-f0ce-4ce3-842a-1b6249233ae9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:20.670377 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a0c711-f0ce-4ce3-842a-1b6249233ae9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:20.672129 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a0c711-f0ce-4ce3-842a-1b6249233ae9 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:20.683359 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 972/1962] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:17:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:17:20.687697 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b6e188-53e9-4299-b9e9-08ebf4d350d1 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:20.700211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b6e188-53e9-4299-b9e9-08ebf4d350d1 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:20.710896 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5467f-1ee9-4dbf-a4e5-0612649ce46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:20.711701 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5467f-1ee9-4dbf-a4e5-0612649ce46b admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:20.712054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5467f-1ee9-4dbf-a4e5-0612649ce46b admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:20.712399 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5467f-1ee9-4dbf-a4e5-0612649ce46b admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:20.713201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5467f-1ee9-4dbf-a4e5-0612649ce46b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:20.714857 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5467f-1ee9-4dbf-a4e5-0612649ce46b admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:20.730779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 991/1963] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:17:20 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:17:20.732632 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a0c711-f0ce-4ce3-842a-1b6249233ae9 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:20.742604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a0c711-f0ce-4ce3-842a-1b6249233ae9 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:20.747209 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983af1e3-cd51-4b4c-9da3-8014a4b090e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:20.748131 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983af1e3-cd51-4b4c-9da3-8014a4b090e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:20.748416 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983af1e3-cd51-4b4c-9da3-8014a4b090e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:20.748689 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983af1e3-cd51-4b4c-9da3-8014a4b090e9 admin admin] PATH_INFO: `/v3/projects/164dfe502ebf4e63ae6bc97e0def2de9/users/e38ed216fecf4bbc903f0bfc5945581f/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:20.763721 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983af1e3-cd51-4b4c-9da3-8014a4b090e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=164dfe502ebf4e63ae6bc97e0def2de9, user_id=e38ed216fecf4bbc903f0bfc5945581f, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:20.766357 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983af1e3-cd51-4b4c-9da3-8014a4b090e9 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:20.781490 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 973/1964] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:17:20 2022] PUT /identity/v3/projects/164dfe502ebf4e63ae6bc97e0def2de9/users/e38ed216fecf4bbc903f0bfc5945581f/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:20.792560 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26907a8a-2e94-431e-a4e6-c54522eadfb3 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:20.793148 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26907a8a-2e94-431e-a4e6-c54522eadfb3 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:20.793148 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26907a8a-2e94-431e-a4e6-c54522eadfb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:20.811983 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:20.820136 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-26907a8a-2e94-431e-a4e6-c54522eadfb3 None None] MFA Rules not processed for user `e38ed216fecf4bbc903f0bfc5945581f`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:20.822077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-26907a8a-2e94-431e-a4e6-c54522eadfb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:20.887759 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 992/1965] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:17:20.899082 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983af1e3-cd51-4b4c-9da3-8014a4b090e9 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:20.900819 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-983af1e3-cd51-4b4c-9da3-8014a4b090e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:20.959216 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983af1e3-cd51-4b4c-9da3-8014a4b090e9 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:20.962875 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d26cfb-1fee-43b5-b66d-b2311731b356 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:20.963532 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d26cfb-1fee-43b5-b66d-b2311731b356 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:20.963818 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d26cfb-1fee-43b5-b66d-b2311731b356 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:20.964098 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d26cfb-1fee-43b5-b66d-b2311731b356 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:20.972888 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d26cfb-1fee-43b5-b66d-b2311731b356 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:20.974465 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d26cfb-1fee-43b5-b66d-b2311731b356 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:20.984223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 974/1966] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:17:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:17:22.769595 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26907a8a-2e94-431e-a4e6-c54522eadfb3 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:22.771093 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-26907a8a-2e94-431e-a4e6-c54522eadfb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:22.834519 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26907a8a-2e94-431e-a4e6-c54522eadfb3 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:22.839260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c387dd0-48fb-4404-98f1-19cb50dbd535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:22.840688 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c387dd0-48fb-4404-98f1-19cb50dbd535 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:22.840973 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c387dd0-48fb-4404-98f1-19cb50dbd535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:22.841249 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c387dd0-48fb-4404-98f1-19cb50dbd535 admin admin] PATH_INFO: `/v3/users/c39eec238b4a4da783d8cc8b700b688f` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:22.842448 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c387dd0-48fb-4404-98f1-19cb50dbd535 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c39eec238b4a4da783d8cc8b700b688f)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:22.844400 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c387dd0-48fb-4404-98f1-19cb50dbd535 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:22.892223 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-2c387dd0-48fb-4404-98f1-19cb50dbd535 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c39eec238b4a4da783d8cc8b700b688f'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:22.899818 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-2c387dd0-48fb-4404-98f1-19cb50dbd535 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c39eec238b4a4da783d8cc8b700b688f'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:22.913048 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-2c387dd0-48fb-4404-98f1-19cb50dbd535 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c39eec238b4a4da783d8cc8b700b688f'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:22.920790 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-2c387dd0-48fb-4404-98f1-19cb50dbd535 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c39eec238b4a4da783d8cc8b700b688f'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:22.921519 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-2c387dd0-48fb-4404-98f1-19cb50dbd535 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c39eec238b4a4da783d8cc8b700b688f'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:22.927490 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 993/1967] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:17:22 2022] DELETE /identity/v3/users/c39eec238b4a4da783d8cc8b700b688f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:22.931885 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6ce27-0d0c-420f-91a8-d2ab24dbbf8e None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:22.932348 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6ce27-0d0c-420f-91a8-d2ab24dbbf8e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:22.932437 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6ce27-0d0c-420f-91a8-d2ab24dbbf8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:22.949583 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-e0b6ce27-0d0c-420f-91a8-d2ab24dbbf8e None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:22.950797 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e0b6ce27-0d0c-420f-91a8-d2ab24dbbf8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:22.983631 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:23.011811 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 975/1968] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:17:23.021712 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c387dd0-48fb-4404-98f1-19cb50dbd535 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:23.023139 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-2c387dd0-48fb-4404-98f1-19cb50dbd535 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:23.046865 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:23.087863 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c387dd0-48fb-4404-98f1-19cb50dbd535 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:23.092247 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22a176-5571-4b99-9f4c-4917f5a92092 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:23.093033 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22a176-5571-4b99-9f4c-4917f5a92092 service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:23.093322 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22a176-5571-4b99-9f4c-4917f5a92092 service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:23.093632 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22a176-5571-4b99-9f4c-4917f5a92092 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:23.102876 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef22a176-5571-4b99-9f4c-4917f5a92092 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:23.104344 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef22a176-5571-4b99-9f4c-4917f5a92092 service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:23.114429 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 994/1969] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:17:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:17:23.406221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b6ce27-0d0c-420f-91a8-d2ab24dbbf8e None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:23.407721 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e0b6ce27-0d0c-420f-91a8-d2ab24dbbf8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:23.421779 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b6ce27-0d0c-420f-91a8-d2ab24dbbf8e None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:23.426653 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a84cb12-88fc-4442-8d22-5be06aa757a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:23.427559 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a84cb12-88fc-4442-8d22-5be06aa757a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:23.427852 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a84cb12-88fc-4442-8d22-5be06aa757a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:23.428125 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a84cb12-88fc-4442-8d22-5be06aa757a0 admin admin] PATH_INFO: `/v3/projects/ad0d5a6f3ef44bcabda6a2b47b99ee58` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:23.429842 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a84cb12-88fc-4442-8d22-5be06aa757a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad0d5a6f3ef44bcabda6a2b47b99ee58)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:23.431715 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a84cb12-88fc-4442-8d22-5be06aa757a0 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:23.488720 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1a84cb12-88fc-4442-8d22-5be06aa757a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad0d5a6f3ef44bcabda6a2b47b99ee58', 'request_id': 'req-1a84cb12-88fc-4442-8d22-5be06aa757a0'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:23.495038 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1a84cb12-88fc-4442-8d22-5be06aa757a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad0d5a6f3ef44bcabda6a2b47b99ee58', 'request_id': 'req-1a84cb12-88fc-4442-8d22-5be06aa757a0'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:23.502756 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1a84cb12-88fc-4442-8d22-5be06aa757a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad0d5a6f3ef44bcabda6a2b47b99ee58', 'request_id': 'req-1a84cb12-88fc-4442-8d22-5be06aa757a0'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:23.515015 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1a84cb12-88fc-4442-8d22-5be06aa757a0 admin admin] The token cache is being invalidate because project ad0d5a6f3ef44bcabda6a2b47b99ee58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:17:23.515171 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1a84cb12-88fc-4442-8d22-5be06aa757a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:23.515753 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1a84cb12-88fc-4442-8d22-5be06aa757a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:23.520399 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 976/1970] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:17:23 2022] DELETE /identity/v3/projects/ad0d5a6f3ef44bcabda6a2b47b99ee58 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:23.521367 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22a176-5571-4b99-9f4c-4917f5a92092 service neutron] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:23.533154 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22a176-5571-4b99-9f4c-4917f5a92092 service neutron] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:23.595645 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa89150-232a-47d8-9256-3187a35a1882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:23.596526 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa89150-232a-47d8-9256-3187a35a1882 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:23.596904 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa89150-232a-47d8-9256-3187a35a1882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:23.597082 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa89150-232a-47d8-9256-3187a35a1882 admin admin] PATH_INFO: `/v3/users/9ae47c0d22014f7bb898577637b4bbac` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:23.598271 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa89150-232a-47d8-9256-3187a35a1882 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ae47c0d22014f7bb898577637b4bbac)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:23.600205 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa89150-232a-47d8-9256-3187a35a1882 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:23.649660 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0aa89150-232a-47d8-9256-3187a35a1882 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ae47c0d22014f7bb898577637b4bbac'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:23.657954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0aa89150-232a-47d8-9256-3187a35a1882 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ae47c0d22014f7bb898577637b4bbac'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:23.671253 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0aa89150-232a-47d8-9256-3187a35a1882 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ae47c0d22014f7bb898577637b4bbac'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:23.678285 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0aa89150-232a-47d8-9256-3187a35a1882 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ae47c0d22014f7bb898577637b4bbac'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:23.678975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0aa89150-232a-47d8-9256-3187a35a1882 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ae47c0d22014f7bb898577637b4bbac'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:23.685265 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 995/1971] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:17:23 2022] DELETE /identity/v3/users/9ae47c0d22014f7bb898577637b4bbac => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:24.033719 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a84cb12-88fc-4442-8d22-5be06aa757a0 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:24.035346 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1a84cb12-88fc-4442-8d22-5be06aa757a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:24.049652 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a84cb12-88fc-4442-8d22-5be06aa757a0 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:24.104852 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a95e6-4efe-48cc-9278-28a1b76db4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:24.106215 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a95e6-4efe-48cc-9278-28a1b76db4e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:24.106486 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a95e6-4efe-48cc-9278-28a1b76db4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:24.106760 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a95e6-4efe-48cc-9278-28a1b76db4e7 admin admin] PATH_INFO: `/v3/projects/a933a460f53b482a8e13a39b05bca43c` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:24.107946 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808a95e6-4efe-48cc-9278-28a1b76db4e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a933a460f53b482a8e13a39b05bca43c)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:24.109852 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808a95e6-4efe-48cc-9278-28a1b76db4e7 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:24.154278 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-808a95e6-4efe-48cc-9278-28a1b76db4e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a933a460f53b482a8e13a39b05bca43c', 'request_id': 'req-808a95e6-4efe-48cc-9278-28a1b76db4e7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:24.160119 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-808a95e6-4efe-48cc-9278-28a1b76db4e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a933a460f53b482a8e13a39b05bca43c', 'request_id': 'req-808a95e6-4efe-48cc-9278-28a1b76db4e7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:24.166870 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-808a95e6-4efe-48cc-9278-28a1b76db4e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a933a460f53b482a8e13a39b05bca43c', 'request_id': 'req-808a95e6-4efe-48cc-9278-28a1b76db4e7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:24.169358 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:24.179871 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-808a95e6-4efe-48cc-9278-28a1b76db4e7 admin admin] The token cache is being invalidate because project a933a460f53b482a8e13a39b05bca43c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:17:24.180158 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-808a95e6-4efe-48cc-9278-28a1b76db4e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:24.180768 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-808a95e6-4efe-48cc-9278-28a1b76db4e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:24.182392 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 977/1972] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:17:24 2022] DELETE /identity/v3/projects/a933a460f53b482a8e13a39b05bca43c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:24.186333 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa89150-232a-47d8-9256-3187a35a1882 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:24.187795 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-0aa89150-232a-47d8-9256-3187a35a1882 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:24.200270 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa89150-232a-47d8-9256-3187a35a1882 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:24.240676 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:24.259793 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd40efa-a63c-4cfa-bb6a-1e8bb737d0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:24.260792 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd40efa-a63c-4cfa-bb6a-1e8bb737d0cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:24.261080 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd40efa-a63c-4cfa-bb6a-1e8bb737d0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:24.261359 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd40efa-a63c-4cfa-bb6a-1e8bb737d0cc admin admin] PATH_INFO: `/v3/users/c5681e361dcb44d18cafe8c5cc0a5d73` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:24.262580 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd40efa-a63c-4cfa-bb6a-1e8bb737d0cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5681e361dcb44d18cafe8c5cc0a5d73)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:24.264531 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd40efa-a63c-4cfa-bb6a-1e8bb737d0cc admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:24.313485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-ecd40efa-a63c-4cfa-bb6a-1e8bb737d0cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5681e361dcb44d18cafe8c5cc0a5d73'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:24.322253 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-ecd40efa-a63c-4cfa-bb6a-1e8bb737d0cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5681e361dcb44d18cafe8c5cc0a5d73'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:24.335529 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-ecd40efa-a63c-4cfa-bb6a-1e8bb737d0cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5681e361dcb44d18cafe8c5cc0a5d73'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:24.343391 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-ecd40efa-a63c-4cfa-bb6a-1e8bb737d0cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5681e361dcb44d18cafe8c5cc0a5d73'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:24.344142 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-ecd40efa-a63c-4cfa-bb6a-1e8bb737d0cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5681e361dcb44d18cafe8c5cc0a5d73'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:24.350072 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 996/1973] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:17:24 2022] DELETE /identity/v3/users/c5681e361dcb44d18cafe8c5cc0a5d73 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:24.696491 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a95e6-4efe-48cc-9278-28a1b76db4e7 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:24.698315 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-808a95e6-4efe-48cc-9278-28a1b76db4e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:24.711991 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a95e6-4efe-48cc-9278-28a1b76db4e7 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:24.769503 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e17d7d4-36a4-40be-9cf2-d7a779d65c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:24.770368 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17d7d4-36a4-40be-9cf2-d7a779d65c17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:24.770653 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17d7d4-36a4-40be-9cf2-d7a779d65c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:24.770962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17d7d4-36a4-40be-9cf2-d7a779d65c17 admin admin] PATH_INFO: `/v3/projects/e878dea8fb424003affbc951848ff041` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:24.772184 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e17d7d4-36a4-40be-9cf2-d7a779d65c17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e878dea8fb424003affbc951848ff041)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:24.774101 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e17d7d4-36a4-40be-9cf2-d7a779d65c17 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:24.822477 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-8e17d7d4-36a4-40be-9cf2-d7a779d65c17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e878dea8fb424003affbc951848ff041', 'request_id': 'req-8e17d7d4-36a4-40be-9cf2-d7a779d65c17'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:24.830559 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-8e17d7d4-36a4-40be-9cf2-d7a779d65c17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e878dea8fb424003affbc951848ff041', 'request_id': 'req-8e17d7d4-36a4-40be-9cf2-d7a779d65c17'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:24.838442 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-8e17d7d4-36a4-40be-9cf2-d7a779d65c17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e878dea8fb424003affbc951848ff041', 'request_id': 'req-8e17d7d4-36a4-40be-9cf2-d7a779d65c17'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:24.850266 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-8e17d7d4-36a4-40be-9cf2-d7a779d65c17 admin admin] The token cache is being invalidate because project e878dea8fb424003affbc951848ff041 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:17:24.850645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-8e17d7d4-36a4-40be-9cf2-d7a779d65c17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:24.851418 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-8e17d7d4-36a4-40be-9cf2-d7a779d65c17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:24.853026 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 978/1974] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:17:24 2022] DELETE /identity/v3/projects/e878dea8fb424003affbc951848ff041 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:24.857794 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ddf2b-8398-46f3-a259-0ba282ad6e76 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:24.858086 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ddf2b-8398-46f3-a259-0ba282ad6e76 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:24.858368 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ddf2b-8398-46f3-a259-0ba282ad6e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:24.876733 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-9d1ddf2b-8398-46f3-a259-0ba282ad6e76 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:24.877950 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-9d1ddf2b-8398-46f3-a259-0ba282ad6e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:24.936250 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 997/1975] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:17:24.943992 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa9b3a-0dd8-4c5b-868d-62698285504e None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:24.944283 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa9b3a-0dd8-4c5b-868d-62698285504e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:24.944562 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa9b3a-0dd8-4c5b-868d-62698285504e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:24.962923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-7daa9b3a-0dd8-4c5b-868d-62698285504e None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:24.964144 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7daa9b3a-0dd8-4c5b-868d-62698285504e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:24.971836 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 979/1976] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:17:24.975890 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ddf2b-8398-46f3-a259-0ba282ad6e76 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:24.985499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ddf2b-8398-46f3-a259-0ba282ad6e76 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:24.990068 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b609d63c-ffc2-4b2a-aff0-3c5f62cdc2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:24.990726 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b609d63c-ffc2-4b2a-aff0-3c5f62cdc2c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:24.991016 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b609d63c-ffc2-4b2a-aff0-3c5f62cdc2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:24.991321 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b609d63c-ffc2-4b2a-aff0-3c5f62cdc2c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:24.992043 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b609d63c-ffc2-4b2a-aff0-3c5f62cdc2c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:17:24.992353 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b609d63c-ffc2-4b2a-aff0-3c5f62cdc2c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:24.993942 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b609d63c-ffc2-4b2a-aff0-3c5f62cdc2c2 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:25.005740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 998/1977] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:17:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:17:25.009805 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa9b3a-0dd8-4c5b-868d-62698285504e None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:25.020068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa9b3a-0dd8-4c5b-868d-62698285504e None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:25.025003 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61577e-d875-40c4-ab8d-94f5ffcf58ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:25.025658 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61577e-d875-40c4-ab8d-94f5ffcf58ae admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:25.025971 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61577e-d875-40c4-ab8d-94f5ffcf58ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:25.026256 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61577e-d875-40c4-ab8d-94f5ffcf58ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:25.027219 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb61577e-d875-40c4-ab8d-94f5ffcf58ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:25.029189 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb61577e-d875-40c4-ab8d-94f5ffcf58ae admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:25.057527 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 980/1978] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:17:25 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:17:25.062100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b609d63c-ffc2-4b2a-aff0-3c5f62cdc2c2 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:25.074302 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b609d63c-ffc2-4b2a-aff0-3c5f62cdc2c2 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:25.124260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:25.138701 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05f75c-ceef-4211-92bf-71721c37f151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:25.139419 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05f75c-ceef-4211-92bf-71721c37f151 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:25.139708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05f75c-ceef-4211-92bf-71721c37f151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:25.139983 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05f75c-ceef-4211-92bf-71721c37f151 admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:25.140718 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05f75c-ceef-4211-92bf-71721c37f151 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:25.142649 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05f75c-ceef-4211-92bf-71721c37f151 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:25.166115 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 999/1979] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:17:25 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:17:25.167406 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61577e-d875-40c4-ab8d-94f5ffcf58ae admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:25.177963 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61577e-d875-40c4-ab8d-94f5ffcf58ae admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:25.182852 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11bcd3-696c-4356-b9c2-d90dbc64c0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:25.183544 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11bcd3-696c-4356-b9c2-d90dbc64c0ca admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:25.184347 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11bcd3-696c-4356-b9c2-d90dbc64c0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:25.184630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11bcd3-696c-4356-b9c2-d90dbc64c0ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:25.185362 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a11bcd3-696c-4356-b9c2-d90dbc64c0ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:25.186972 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a11bcd3-696c-4356-b9c2-d90dbc64c0ca admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:25.198478 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 981/1980] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:17:25.203012 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05f75c-ceef-4211-92bf-71721c37f151 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:25.212863 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05f75c-ceef-4211-92bf-71721c37f151 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:25.217466 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fcd6ed-20e3-4091-a33b-f1962ccced85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:25.218120 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fcd6ed-20e3-4091-a33b-f1962ccced85 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:25.218405 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fcd6ed-20e3-4091-a33b-f1962ccced85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:25.218683 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fcd6ed-20e3-4091-a33b-f1962ccced85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:25.219438 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fcd6ed-20e3-4091-a33b-f1962ccced85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:25.221011 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fcd6ed-20e3-4091-a33b-f1962ccced85 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:25.231384 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1000/1981] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:17:25.235449 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11bcd3-696c-4356-b9c2-d90dbc64c0ca admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:25.245460 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11bcd3-696c-4356-b9c2-d90dbc64c0ca admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:25.250575 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc37550-e2ed-4d2d-b0fd-7740990dc951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:25.251500 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc37550-e2ed-4d2d-b0fd-7740990dc951 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:25.251781 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc37550-e2ed-4d2d-b0fd-7740990dc951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:25.252056 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc37550-e2ed-4d2d-b0fd-7740990dc951 admin admin] PATH_INFO: `/v3/projects/d53fbaa10ec74ac3a29413d55e1b68d1/users/5e00ed946d9040f1b9eac99d128b48a7/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:25.266178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc37550-e2ed-4d2d-b0fd-7740990dc951 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d53fbaa10ec74ac3a29413d55e1b68d1, user_id=5e00ed946d9040f1b9eac99d128b48a7, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:25.268841 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc37550-e2ed-4d2d-b0fd-7740990dc951 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:25.281896 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 982/1982] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:17:25 2022] PUT /identity/v3/projects/d53fbaa10ec74ac3a29413d55e1b68d1/users/5e00ed946d9040f1b9eac99d128b48a7/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:17:25.291849 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610727f-13b6-428b-84d4-ff8db3a4a14d None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:25.292149 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610727f-13b6-428b-84d4-ff8db3a4a14d None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:25.292432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610727f-13b6-428b-84d4-ff8db3a4a14d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:25.318907 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-4610727f-13b6-428b-84d4-ff8db3a4a14d None None] MFA Rules not processed for user `5e00ed946d9040f1b9eac99d128b48a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:25.320140 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-4610727f-13b6-428b-84d4-ff8db3a4a14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:25.382810 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1001/1983] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:17:25.387424 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc37550-e2ed-4d2d-b0fd-7740990dc951 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:25.410850 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:25.463479 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc37550-e2ed-4d2d-b0fd-7740990dc951 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:25.468266 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d39ed2-f3a4-4a64-9a82-4434cde8c814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:25.468926 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d39ed2-f3a4-4a64-9a82-4434cde8c814 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:25.469211 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d39ed2-f3a4-4a64-9a82-4434cde8c814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:25.469487 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d39ed2-f3a4-4a64-9a82-4434cde8c814 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:25.470189 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d39ed2-f3a4-4a64-9a82-4434cde8c814 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:25.472154 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d39ed2-f3a4-4a64-9a82-4434cde8c814 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:25.498164 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 983/1984] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:17:25 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:17:25.502154 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4610727f-13b6-428b-84d4-ff8db3a4a14d None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:25.512656 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4610727f-13b6-428b-84d4-ff8db3a4a14d None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:25.567637 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223dd776-c7de-4d0c-bd82-c782edc19d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:25.568304 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223dd776-c7de-4d0c-bd82-c782edc19d71 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:25.568590 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223dd776-c7de-4d0c-bd82-c782edc19d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:25.568869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223dd776-c7de-4d0c-bd82-c782edc19d71 admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:25.569566 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223dd776-c7de-4d0c-bd82-c782edc19d71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:25.571534 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223dd776-c7de-4d0c-bd82-c782edc19d71 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:25.591601 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1002/1985] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:17:25 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:17:25.595506 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d39ed2-f3a4-4a64-9a82-4434cde8c814 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:25.605574 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d39ed2-f3a4-4a64-9a82-4434cde8c814 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:25.610122 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661bc379-f167-475a-9c03-a27fd448a889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:25.610773 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661bc379-f167-475a-9c03-a27fd448a889 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:25.611061 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661bc379-f167-475a-9c03-a27fd448a889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:25.611367 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661bc379-f167-475a-9c03-a27fd448a889 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:25.612125 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661bc379-f167-475a-9c03-a27fd448a889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:25.613716 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661bc379-f167-475a-9c03-a27fd448a889 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:25.627342 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 984/1986] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:17:25.628502 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223dd776-c7de-4d0c-bd82-c782edc19d71 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:25.638972 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223dd776-c7de-4d0c-bd82-c782edc19d71 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:25.643643 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6095fa5-0f9d-47ac-a3ba-9fadc55a9a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:25.644296 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6095fa5-0f9d-47ac-a3ba-9fadc55a9a12 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:25.644577 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6095fa5-0f9d-47ac-a3ba-9fadc55a9a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:25.644855 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6095fa5-0f9d-47ac-a3ba-9fadc55a9a12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:25.645567 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6095fa5-0f9d-47ac-a3ba-9fadc55a9a12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:25.647222 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6095fa5-0f9d-47ac-a3ba-9fadc55a9a12 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:25.657375 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1003/1987] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:17:25.661175 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661bc379-f167-475a-9c03-a27fd448a889 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:25.670946 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661bc379-f167-475a-9c03-a27fd448a889 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:25.675488 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db3cb1-bd21-462f-93b9-faed4f32fbc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:25.676359 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db3cb1-bd21-462f-93b9-faed4f32fbc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:25.676645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db3cb1-bd21-462f-93b9-faed4f32fbc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:25.676926 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db3cb1-bd21-462f-93b9-faed4f32fbc8 admin admin] PATH_INFO: `/v3/projects/4fafba350bd54dcdb1b278971db2553e/users/1cfef429dd444fc2b043daf4d18ad8b0/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:25.691657 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23db3cb1-bd21-462f-93b9-faed4f32fbc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fafba350bd54dcdb1b278971db2553e, user_id=1cfef429dd444fc2b043daf4d18ad8b0, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:25.694290 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23db3cb1-bd21-462f-93b9-faed4f32fbc8 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:25.708310 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 985/1988] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:17:25 2022] PUT /identity/v3/projects/4fafba350bd54dcdb1b278971db2553e/users/1cfef429dd444fc2b043daf4d18ad8b0/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:25.775783 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9fc7f5-e26f-433c-bbea-b39b3d8b9bf9 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:25.776071 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9fc7f5-e26f-433c-bbea-b39b3d8b9bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:25.776351 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9fc7f5-e26f-433c-bbea-b39b3d8b9bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:25.805395 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-1c9fc7f5-e26f-433c-bbea-b39b3d8b9bf9 None None] MFA Rules not processed for user `1cfef429dd444fc2b043daf4d18ad8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:25.806608 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-1c9fc7f5-e26f-433c-bbea-b39b3d8b9bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:25.830709 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:25.879906 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1004/1989] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:17:25.884412 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db3cb1-bd21-462f-93b9-faed4f32fbc8 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:25.946043 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db3cb1-bd21-462f-93b9-faed4f32fbc8 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:25.952488 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38196c-9fdb-48b6-8160-c23b1525475d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:25.953349 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38196c-9fdb-48b6-8160-c23b1525475d admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:25.953708 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38196c-9fdb-48b6-8160-c23b1525475d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:25.954256 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38196c-9fdb-48b6-8160-c23b1525475d admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:25.955362 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38196c-9fdb-48b6-8160-c23b1525475d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:25.958996 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38196c-9fdb-48b6-8160-c23b1525475d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:25.994953 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 986/1990] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:17:25 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:17:25.996166 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9fc7f5-e26f-433c-bbea-b39b3d8b9bf9 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:26.006853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9fc7f5-e26f-433c-bbea-b39b3d8b9bf9 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:26.064537 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef41651c-6466-4b3b-b67e-2e0bee73deb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:26.065203 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41651c-6466-4b3b-b67e-2e0bee73deb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:26.065485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41651c-6466-4b3b-b67e-2e0bee73deb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:26.065758 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41651c-6466-4b3b-b67e-2e0bee73deb8 admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:26.066438 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef41651c-6466-4b3b-b67e-2e0bee73deb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:26.068598 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef41651c-6466-4b3b-b67e-2e0bee73deb8 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:26.093423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1005/1991] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:17:25 2022] POST /identity/v3/users => generated 298 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:17:26.094983 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38196c-9fdb-48b6-8160-c23b1525475d admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:26.106363 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38196c-9fdb-48b6-8160-c23b1525475d admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:26.111706 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98fc958-2756-4b10-a493-2702ebe0d6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:26.112491 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98fc958-2756-4b10-a493-2702ebe0d6cb admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:26.112610 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98fc958-2756-4b10-a493-2702ebe0d6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:26.113011 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98fc958-2756-4b10-a493-2702ebe0d6cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:26.113835 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98fc958-2756-4b10-a493-2702ebe0d6cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:26.115459 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98fc958-2756-4b10-a493-2702ebe0d6cb admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:26.127209 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 987/1992] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:17:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:17:26.132061 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef41651c-6466-4b3b-b67e-2e0bee73deb8 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:26.144748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef41651c-6466-4b3b-b67e-2e0bee73deb8 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:26.149422 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505348ee-f6d1-4ba1-a197-d2dcd581a5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:26.150306 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505348ee-f6d1-4ba1-a197-d2dcd581a5f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:26.150592 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505348ee-f6d1-4ba1-a197-d2dcd581a5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:26.150871 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505348ee-f6d1-4ba1-a197-d2dcd581a5f3 admin admin] PATH_INFO: `/v3/projects/fc781647cd484c83938e6e279fdc97ea/users/ee68fcf9434541e1a3e8aa7d1c0ab57c/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:26.166491 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505348ee-f6d1-4ba1-a197-d2dcd581a5f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc781647cd484c83938e6e279fdc97ea, user_id=ee68fcf9434541e1a3e8aa7d1c0ab57c, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:26.169159 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505348ee-f6d1-4ba1-a197-d2dcd581a5f3 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:26.182999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1006/1993] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:17:26 2022] PUT /identity/v3/projects/fc781647cd484c83938e6e279fdc97ea/users/ee68fcf9434541e1a3e8aa7d1c0ab57c/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:26.186862 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98fc958-2756-4b10-a493-2702ebe0d6cb admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:26.196806 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98fc958-2756-4b10-a493-2702ebe0d6cb admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:26.243849 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:26.265611 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c008ea3-5473-4d7f-bf68-f3a29f1cb8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:26.266281 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c008ea3-5473-4d7f-bf68-f3a29f1cb8d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:26.266594 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c008ea3-5473-4d7f-bf68-f3a29f1cb8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:26.266870 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c008ea3-5473-4d7f-bf68-f3a29f1cb8d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:26.267651 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c008ea3-5473-4d7f-bf68-f3a29f1cb8d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:26.269788 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c008ea3-5473-4d7f-bf68-f3a29f1cb8d9 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:26.288978 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 988/1994] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:17:26 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:17:26.290277 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505348ee-f6d1-4ba1-a197-d2dcd581a5f3 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:26.301546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505348ee-f6d1-4ba1-a197-d2dcd581a5f3 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:26.306185 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c69f1-7acf-4e3d-b0ee-4c17da845742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:26.307074 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c69f1-7acf-4e3d-b0ee-4c17da845742 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:26.307387 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c69f1-7acf-4e3d-b0ee-4c17da845742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:26.307777 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c69f1-7acf-4e3d-b0ee-4c17da845742 admin admin] PATH_INFO: `/v3/projects/fc781647cd484c83938e6e279fdc97ea/users/ee68fcf9434541e1a3e8aa7d1c0ab57c/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:26.309757 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01c69f1-7acf-4e3d-b0ee-4c17da845742 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc781647cd484c83938e6e279fdc97ea, user_id=ee68fcf9434541e1a3e8aa7d1c0ab57c, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:26.312406 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01c69f1-7acf-4e3d-b0ee-4c17da845742 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:26.322775 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1007/1995] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:17:26 2022] PUT /identity/v3/projects/fc781647cd484c83938e6e279fdc97ea/users/ee68fcf9434541e1a3e8aa7d1c0ab57c/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:26.333716 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e7d023-5cdf-43fc-9c05-9ecef6396009 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:26.334031 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e7d023-5cdf-43fc-9c05-9ecef6396009 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:26.334716 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e7d023-5cdf-43fc-9c05-9ecef6396009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:26.365639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-f3e7d023-5cdf-43fc-9c05-9ecef6396009 None None] MFA Rules not processed for user `ee68fcf9434541e1a3e8aa7d1c0ab57c`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:26.366842 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f3e7d023-5cdf-43fc-9c05-9ecef6396009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:26.432342 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 989/1996] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:26 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:17:26.443746 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c69f1-7acf-4e3d-b0ee-4c17da845742 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:26.445425 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-b01c69f1-7acf-4e3d-b0ee-4c17da845742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:26.504538 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c69f1-7acf-4e3d-b0ee-4c17da845742 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:26.508179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673372c5-2b21-47cb-8c64-38950f151180 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:26.508828 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673372c5-2b21-47cb-8c64-38950f151180 service cinder] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:26.509116 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673372c5-2b21-47cb-8c64-38950f151180 service cinder] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:26.509396 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673372c5-2b21-47cb-8c64-38950f151180 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:26.539667 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:26.566992 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673372c5-2b21-47cb-8c64-38950f151180 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:26.568506 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673372c5-2b21-47cb-8c64-38950f151180 service cinder] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:26.578851 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1008/1997] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:17:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:17:29.155069 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e7d023-5cdf-43fc-9c05-9ecef6396009 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:29.156602 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f3e7d023-5cdf-43fc-9c05-9ecef6396009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:29.168251 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e7d023-5cdf-43fc-9c05-9ecef6396009 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:29.171812 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f98d17-4022-4fec-9c24-bce55b1c0745 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:29.172451 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f98d17-4022-4fec-9c24-bce55b1c0745 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:29.172736 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f98d17-4022-4fec-9c24-bce55b1c0745 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:29.173011 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f98d17-4022-4fec-9c24-bce55b1c0745 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:29.235498 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f98d17-4022-4fec-9c24-bce55b1c0745 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:29.237061 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f98d17-4022-4fec-9c24-bce55b1c0745 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:29.246488 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 990/1998] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:17:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:17:32.367686 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673372c5-2b21-47cb-8c64-38950f151180 service cinder] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:32.376410 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673372c5-2b21-47cb-8c64-38950f151180 service cinder] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:32.380009 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6073b74-9608-4d49-8e8f-d23dd8ef4d33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:32.380641 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6073b74-9608-4d49-8e8f-d23dd8ef4d33 service cinder] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:32.380921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6073b74-9608-4d49-8e8f-d23dd8ef4d33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:32.381196 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6073b74-9608-4d49-8e8f-d23dd8ef4d33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:32.390451 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6073b74-9608-4d49-8e8f-d23dd8ef4d33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:32.391914 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6073b74-9608-4d49-8e8f-d23dd8ef4d33 service cinder] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:32.401702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1009/1999] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:17:32 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:17:35.661144 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f98d17-4022-4fec-9c24-bce55b1c0745 service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:35.662995 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e9f98d17-4022-4fec-9c24-bce55b1c0745 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:35.739728 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f98d17-4022-4fec-9c24-bce55b1c0745 service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:35.745426 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a986104-9a68-4a93-af57-e75180cb900e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:35.746317 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a986104-9a68-4a93-af57-e75180cb900e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:35.746669 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a986104-9a68-4a93-af57-e75180cb900e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:35.747080 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a986104-9a68-4a93-af57-e75180cb900e admin admin] PATH_INFO: `/v3/users/6afcba283b764ee69f8e3335908f6d71` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:35.748531 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a986104-9a68-4a93-af57-e75180cb900e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6afcba283b764ee69f8e3335908f6d71)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:35.750755 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a986104-9a68-4a93-af57-e75180cb900e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:35.753652 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:35.816698 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4a986104-9a68-4a93-af57-e75180cb900e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6afcba283b764ee69f8e3335908f6d71'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:35.827446 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4a986104-9a68-4a93-af57-e75180cb900e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6afcba283b764ee69f8e3335908f6d71'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:35.846083 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4a986104-9a68-4a93-af57-e75180cb900e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6afcba283b764ee69f8e3335908f6d71'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:35.846880 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4a986104-9a68-4a93-af57-e75180cb900e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6afcba283b764ee69f8e3335908f6d71'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:35.854686 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4a986104-9a68-4a93-af57-e75180cb900e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6afcba283b764ee69f8e3335908f6d71'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:35.861402 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 991/2000] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:17:35 2022] DELETE /identity/v3/users/6afcba283b764ee69f8e3335908f6d71 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:35.865933 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b790f47-6f57-48ef-83ee-1ca16ab04282 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:35.867143 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b790f47-6f57-48ef-83ee-1ca16ab04282 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:35.867617 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b790f47-6f57-48ef-83ee-1ca16ab04282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:35.891515 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-3b790f47-6f57-48ef-83ee-1ca16ab04282 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:35.892820 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-3b790f47-6f57-48ef-83ee-1ca16ab04282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:35.985727 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1010/2001] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:17:35.994237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a986104-9a68-4a93-af57-e75180cb900e admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:35.996898 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4a986104-9a68-4a93-af57-e75180cb900e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:36.060732 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a986104-9a68-4a93-af57-e75180cb900e admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:36.064467 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f720355-c9f0-4262-84cd-35e24dbd8e5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:36.065144 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f720355-c9f0-4262-84cd-35e24dbd8e5d service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:36.065425 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f720355-c9f0-4262-84cd-35e24dbd8e5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:36.065700 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f720355-c9f0-4262-84cd-35e24dbd8e5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:36.076994 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f720355-c9f0-4262-84cd-35e24dbd8e5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:36.078483 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f720355-c9f0-4262-84cd-35e24dbd8e5d service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:36.088380 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 992/2002] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:17:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:17:36.478264 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b790f47-6f57-48ef-83ee-1ca16ab04282 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:36.480748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-3b790f47-6f57-48ef-83ee-1ca16ab04282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:36.520415 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b790f47-6f57-48ef-83ee-1ca16ab04282 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:36.527992 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318bc284-583c-4c61-af22-8828b30fd6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:36.529171 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318bc284-583c-4c61-af22-8828b30fd6f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:36.529374 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318bc284-583c-4c61-af22-8828b30fd6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:36.529684 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318bc284-583c-4c61-af22-8828b30fd6f5 admin admin] PATH_INFO: `/v3/projects/f006fe57505d43bbb0cdfd8d739f385c` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:36.531784 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318bc284-583c-4c61-af22-8828b30fd6f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f006fe57505d43bbb0cdfd8d739f385c)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:36.533871 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318bc284-583c-4c61-af22-8828b30fd6f5 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:36.593372 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-318bc284-583c-4c61-af22-8828b30fd6f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f006fe57505d43bbb0cdfd8d739f385c', 'request_id': 'req-318bc284-583c-4c61-af22-8828b30fd6f5'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:36.601266 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-318bc284-583c-4c61-af22-8828b30fd6f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f006fe57505d43bbb0cdfd8d739f385c', 'request_id': 'req-318bc284-583c-4c61-af22-8828b30fd6f5'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:36.610385 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-318bc284-583c-4c61-af22-8828b30fd6f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f006fe57505d43bbb0cdfd8d739f385c', 'request_id': 'req-318bc284-583c-4c61-af22-8828b30fd6f5'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:36.623611 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-318bc284-583c-4c61-af22-8828b30fd6f5 admin admin] The token cache is being invalidate because project f006fe57505d43bbb0cdfd8d739f385c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:17:36.623909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-318bc284-583c-4c61-af22-8828b30fd6f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:36.625165 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-318bc284-583c-4c61-af22-8828b30fd6f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:36.626932 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1011/2003] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:17:36 2022] DELETE /identity/v3/projects/f006fe57505d43bbb0cdfd8d739f385c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:36.632669 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deded95e-5074-4e24-adb8-27d97ca8b3ab None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:36.632961 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deded95e-5074-4e24-adb8-27d97ca8b3ab None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:36.633248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deded95e-5074-4e24-adb8-27d97ca8b3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:36.652428 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-deded95e-5074-4e24-adb8-27d97ca8b3ab None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:36.653712 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-deded95e-5074-4e24-adb8-27d97ca8b3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:36.717127 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 993/2004] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:17:36.725269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18005410-2320-4a92-b633-f7500b4d3d43 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:36.725564 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18005410-2320-4a92-b633-f7500b4d3d43 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:36.725850 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18005410-2320-4a92-b633-f7500b4d3d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:36.744390 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-18005410-2320-4a92-b633-f7500b4d3d43 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:36.745615 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-18005410-2320-4a92-b633-f7500b4d3d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:36.754374 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1012/2005] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:17:36.758422 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deded95e-5074-4e24-adb8-27d97ca8b3ab None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:36.769130 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deded95e-5074-4e24-adb8-27d97ca8b3ab None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:36.774458 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e73a13c-a96b-4545-8967-66036a971d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:36.775143 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73a13c-a96b-4545-8967-66036a971d5a admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:36.775456 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73a13c-a96b-4545-8967-66036a971d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:36.775734 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73a13c-a96b-4545-8967-66036a971d5a admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:36.776475 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e73a13c-a96b-4545-8967-66036a971d5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:17:36.777048 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e73a13c-a96b-4545-8967-66036a971d5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:36.778720 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e73a13c-a96b-4545-8967-66036a971d5a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:36.807120 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 994/2006] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:17:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:17:36.807269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18005410-2320-4a92-b633-f7500b4d3d43 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:36.813837 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18005410-2320-4a92-b633-f7500b4d3d43 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:36.818566 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3cc76-c6e3-441d-ab25-a19ab33502b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:36.819220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3cc76-c6e3-441d-ab25-a19ab33502b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:36.819541 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3cc76-c6e3-441d-ab25-a19ab33502b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:36.819816 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3cc76-c6e3-441d-ab25-a19ab33502b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:36.820511 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d3cc76-c6e3-441d-ab25-a19ab33502b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:36.822392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d3cc76-c6e3-441d-ab25-a19ab33502b7 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:36.851335 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1013/2007] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:17:36 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:17:36.854265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e73a13c-a96b-4545-8967-66036a971d5a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:36.864916 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e73a13c-a96b-4545-8967-66036a971d5a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:36.873431 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:36.926014 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d319464-4c15-4162-968e-3cd6568b776b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:36.926693 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d319464-4c15-4162-968e-3cd6568b776b admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:36.926984 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d319464-4c15-4162-968e-3cd6568b776b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:36.927267 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d319464-4c15-4162-968e-3cd6568b776b admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:36.928007 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d319464-4c15-4162-968e-3cd6568b776b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:36.929935 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d319464-4c15-4162-968e-3cd6568b776b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:36.953111 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 995/2008] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:17:36 2022] POST /identity/v3/users => generated 283 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:17:36.958746 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3cc76-c6e3-441d-ab25-a19ab33502b7 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:36.970202 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3cc76-c6e3-441d-ab25-a19ab33502b7 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:36.976278 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b51248-5b18-4ae6-a73a-d86bba41a19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:36.977221 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b51248-5b18-4ae6-a73a-d86bba41a19c admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:36.977605 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b51248-5b18-4ae6-a73a-d86bba41a19c admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:36.978281 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b51248-5b18-4ae6-a73a-d86bba41a19c admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:36.979700 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b51248-5b18-4ae6-a73a-d86bba41a19c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:36.983017 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b51248-5b18-4ae6-a73a-d86bba41a19c admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:37.004658 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1014/2009] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:17:37.008773 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d319464-4c15-4162-968e-3cd6568b776b admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:37.019197 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d319464-4c15-4162-968e-3cd6568b776b admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:37.024282 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae641de-78eb-47e7-b534-7394f4d08232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:37.024955 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae641de-78eb-47e7-b534-7394f4d08232 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:37.025275 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae641de-78eb-47e7-b534-7394f4d08232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:37.025576 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae641de-78eb-47e7-b534-7394f4d08232 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:37.026279 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae641de-78eb-47e7-b534-7394f4d08232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:37.028855 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae641de-78eb-47e7-b534-7394f4d08232 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:37.042481 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 996/2010] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:17:37.044535 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b51248-5b18-4ae6-a73a-d86bba41a19c admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:37.050558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:37.060333 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b51248-5b18-4ae6-a73a-d86bba41a19c admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:37.068213 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f0d96-7476-4cfa-a84b-aa01ca5f30de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:37.069191 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f0d96-7476-4cfa-a84b-aa01ca5f30de admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:37.069526 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f0d96-7476-4cfa-a84b-aa01ca5f30de admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:37.070330 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f0d96-7476-4cfa-a84b-aa01ca5f30de admin admin] PATH_INFO: `/v3/projects/9aa2047f5cde4b5b930b2518105705de/users/42cbcce54d6b4da3a0bffa1b923a605a/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:37.085790 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347f0d96-7476-4cfa-a84b-aa01ca5f30de admin admin] RBAC: Authorizing `identity:create_grant(project_id=9aa2047f5cde4b5b930b2518105705de, user_id=42cbcce54d6b4da3a0bffa1b923a605a, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:37.088582 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347f0d96-7476-4cfa-a84b-aa01ca5f30de admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:37.102458 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1015/2011] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:17:37 2022] PUT /identity/v3/projects/9aa2047f5cde4b5b930b2518105705de/users/42cbcce54d6b4da3a0bffa1b923a605a/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:17:37.113555 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cbb6a-1fd7-4ea9-9d29-96be27a2d359 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:37.113963 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cbb6a-1fd7-4ea9-9d29-96be27a2d359 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:37.114369 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cbb6a-1fd7-4ea9-9d29-96be27a2d359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:37.141578 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c29cbb6a-1fd7-4ea9-9d29-96be27a2d359 None None] MFA Rules not processed for user `42cbcce54d6b4da3a0bffa1b923a605a`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:37.142775 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c29cbb6a-1fd7-4ea9-9d29-96be27a2d359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:37.211692 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 997/2012] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:37 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:17:37.231003 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f0d96-7476-4cfa-a84b-aa01ca5f30de admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:37.234269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-347f0d96-7476-4cfa-a84b-aa01ca5f30de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:37.300550 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f0d96-7476-4cfa-a84b-aa01ca5f30de admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:37.304203 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a3ff68-1054-4232-8c3d-ba23bdcf3133 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:37.304851 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3ff68-1054-4232-8c3d-ba23bdcf3133 service cinder] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:37.305138 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3ff68-1054-4232-8c3d-ba23bdcf3133 service cinder] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:37.305417 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3ff68-1054-4232-8c3d-ba23bdcf3133 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:37.314296 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a3ff68-1054-4232-8c3d-ba23bdcf3133 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:37.315807 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a3ff68-1054-4232-8c3d-ba23bdcf3133 service cinder] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:37.327020 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1016/2013] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:17:37 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:17:39.553604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29cbb6a-1fd7-4ea9-9d29-96be27a2d359 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:39.555068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c29cbb6a-1fd7-4ea9-9d29-96be27a2d359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:39.628223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29cbb6a-1fd7-4ea9-9d29-96be27a2d359 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:39.632794 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66baa94-f5c7-4c95-b512-aa06d3a7497e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:39.633673 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66baa94-f5c7-4c95-b512-aa06d3a7497e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:39.633974 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66baa94-f5c7-4c95-b512-aa06d3a7497e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:39.634235 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66baa94-f5c7-4c95-b512-aa06d3a7497e admin admin] PATH_INFO: `/v3/users/5e00ed946d9040f1b9eac99d128b48a7` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:39.635480 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66baa94-f5c7-4c95-b512-aa06d3a7497e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e00ed946d9040f1b9eac99d128b48a7)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:39.637938 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66baa94-f5c7-4c95-b512-aa06d3a7497e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:39.659815 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:39.687439 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d66baa94-f5c7-4c95-b512-aa06d3a7497e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e00ed946d9040f1b9eac99d128b48a7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:39.695135 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d66baa94-f5c7-4c95-b512-aa06d3a7497e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e00ed946d9040f1b9eac99d128b48a7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:39.707999 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d66baa94-f5c7-4c95-b512-aa06d3a7497e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e00ed946d9040f1b9eac99d128b48a7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:39.708767 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d66baa94-f5c7-4c95-b512-aa06d3a7497e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e00ed946d9040f1b9eac99d128b48a7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:39.716623 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d66baa94-f5c7-4c95-b512-aa06d3a7497e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e00ed946d9040f1b9eac99d128b48a7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:39.723008 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 998/2014] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:17:39 2022] DELETE /identity/v3/users/5e00ed946d9040f1b9eac99d128b48a7 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:39.727628 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1531ee-b9e7-4c3a-b608-14cc7f9beb0d None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:39.728111 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1531ee-b9e7-4c3a-b608-14cc7f9beb0d None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:39.728204 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1531ee-b9e7-4c3a-b608-14cc7f9beb0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:39.746163 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-8f1531ee-b9e7-4c3a-b608-14cc7f9beb0d None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:39.747783 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-8f1531ee-b9e7-4c3a-b608-14cc7f9beb0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:39.815533 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1017/2015] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:17:39.822151 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66baa94-f5c7-4c95-b512-aa06d3a7497e admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:39.823672 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d66baa94-f5c7-4c95-b512-aa06d3a7497e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:39.900884 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66baa94-f5c7-4c95-b512-aa06d3a7497e admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:39.905000 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738a993c-0b45-4871-b3b7-2fd758f5469d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:39.905663 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a993c-0b45-4871-b3b7-2fd758f5469d service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:39.905950 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a993c-0b45-4871-b3b7-2fd758f5469d service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:39.906227 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a993c-0b45-4871-b3b7-2fd758f5469d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:39.916934 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738a993c-0b45-4871-b3b7-2fd758f5469d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:39.918440 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738a993c-0b45-4871-b3b7-2fd758f5469d service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:39.929115 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 999/2016] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:17:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:17:40.278177 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1531ee-b9e7-4c3a-b608-14cc7f9beb0d None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:40.279729 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-8f1531ee-b9e7-4c3a-b608-14cc7f9beb0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:40.295160 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1531ee-b9e7-4c3a-b608-14cc7f9beb0d None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:40.301097 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eda45a-f454-4b77-86d3-6979754d342e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:40.304645 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eda45a-f454-4b77-86d3-6979754d342e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:40.305187 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eda45a-f454-4b77-86d3-6979754d342e admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:40.305620 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eda45a-f454-4b77-86d3-6979754d342e admin admin] PATH_INFO: `/v3/projects/d53fbaa10ec74ac3a29413d55e1b68d1` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:40.307371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eda45a-f454-4b77-86d3-6979754d342e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d53fbaa10ec74ac3a29413d55e1b68d1)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:40.310419 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eda45a-f454-4b77-86d3-6979754d342e admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:40.374426 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-85eda45a-f454-4b77-86d3-6979754d342e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd53fbaa10ec74ac3a29413d55e1b68d1', 'request_id': 'req-85eda45a-f454-4b77-86d3-6979754d342e'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:40.381380 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-85eda45a-f454-4b77-86d3-6979754d342e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd53fbaa10ec74ac3a29413d55e1b68d1', 'request_id': 'req-85eda45a-f454-4b77-86d3-6979754d342e'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:40.390085 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-85eda45a-f454-4b77-86d3-6979754d342e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd53fbaa10ec74ac3a29413d55e1b68d1', 'request_id': 'req-85eda45a-f454-4b77-86d3-6979754d342e'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:40.404255 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-85eda45a-f454-4b77-86d3-6979754d342e admin admin] The token cache is being invalidate because project d53fbaa10ec74ac3a29413d55e1b68d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:17:40.404592 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-85eda45a-f454-4b77-86d3-6979754d342e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:40.405317 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-85eda45a-f454-4b77-86d3-6979754d342e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:40.410017 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1018/2017] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:17:40 2022] DELETE /identity/v3/projects/d53fbaa10ec74ac3a29413d55e1b68d1 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:40.411280 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738a993c-0b45-4871-b3b7-2fd758f5469d service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:40.422347 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738a993c-0b45-4871-b3b7-2fd758f5469d service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:40.541020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56781ca5-ccd4-4eda-8038-cfc9f34161a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:40.542290 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56781ca5-ccd4-4eda-8038-cfc9f34161a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:40.542396 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56781ca5-ccd4-4eda-8038-cfc9f34161a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:40.542646 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56781ca5-ccd4-4eda-8038-cfc9f34161a3 admin admin] PATH_INFO: `/v3/users/1cfef429dd444fc2b043daf4d18ad8b0` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:40.544028 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56781ca5-ccd4-4eda-8038-cfc9f34161a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cfef429dd444fc2b043daf4d18ad8b0)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:40.547014 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56781ca5-ccd4-4eda-8038-cfc9f34161a3 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:40.607130 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:40.611841 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-56781ca5-ccd4-4eda-8038-cfc9f34161a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cfef429dd444fc2b043daf4d18ad8b0'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:40.623416 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-56781ca5-ccd4-4eda-8038-cfc9f34161a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cfef429dd444fc2b043daf4d18ad8b0'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:40.640581 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-56781ca5-ccd4-4eda-8038-cfc9f34161a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cfef429dd444fc2b043daf4d18ad8b0'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:40.641330 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-56781ca5-ccd4-4eda-8038-cfc9f34161a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cfef429dd444fc2b043daf4d18ad8b0'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:40.655397 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-56781ca5-ccd4-4eda-8038-cfc9f34161a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cfef429dd444fc2b043daf4d18ad8b0'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:40.663383 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1000/2018] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:17:40 2022] DELETE /identity/v3/users/1cfef429dd444fc2b043daf4d18ad8b0 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:40.963111 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eda45a-f454-4b77-86d3-6979754d342e admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:40.964764 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-85eda45a-f454-4b77-86d3-6979754d342e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:40.977593 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:40.983014 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eda45a-f454-4b77-86d3-6979754d342e admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:41.042314 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c62b01-946d-47e3-9928-3fd62c5ba7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:41.043173 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c62b01-946d-47e3-9928-3fd62c5ba7d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:41.044013 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c62b01-946d-47e3-9928-3fd62c5ba7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:41.044289 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c62b01-946d-47e3-9928-3fd62c5ba7d1 admin admin] PATH_INFO: `/v3/projects/4fafba350bd54dcdb1b278971db2553e` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:41.045451 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c62b01-946d-47e3-9928-3fd62c5ba7d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fafba350bd54dcdb1b278971db2553e)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:41.047494 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c62b01-946d-47e3-9928-3fd62c5ba7d1 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:41.094590 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-08c62b01-946d-47e3-9928-3fd62c5ba7d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fafba350bd54dcdb1b278971db2553e', 'request_id': 'req-08c62b01-946d-47e3-9928-3fd62c5ba7d1'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:41.101781 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-08c62b01-946d-47e3-9928-3fd62c5ba7d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fafba350bd54dcdb1b278971db2553e', 'request_id': 'req-08c62b01-946d-47e3-9928-3fd62c5ba7d1'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:41.109638 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-08c62b01-946d-47e3-9928-3fd62c5ba7d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fafba350bd54dcdb1b278971db2553e', 'request_id': 'req-08c62b01-946d-47e3-9928-3fd62c5ba7d1'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:41.119903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-08c62b01-946d-47e3-9928-3fd62c5ba7d1 admin admin] The token cache is being invalidate because project 4fafba350bd54dcdb1b278971db2553e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:17:41.120189 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-08c62b01-946d-47e3-9928-3fd62c5ba7d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:41.120809 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-08c62b01-946d-47e3-9928-3fd62c5ba7d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:41.122947 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1019/2019] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:17:40 2022] DELETE /identity/v3/projects/4fafba350bd54dcdb1b278971db2553e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:41.127013 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56781ca5-ccd4-4eda-8038-cfc9f34161a3 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:41.128495 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-56781ca5-ccd4-4eda-8038-cfc9f34161a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:41.141165 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56781ca5-ccd4-4eda-8038-cfc9f34161a3 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:41.200463 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff6301e-7e44-4177-8f27-c2a55d89055f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:41.201344 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6301e-7e44-4177-8f27-c2a55d89055f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:41.201634 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6301e-7e44-4177-8f27-c2a55d89055f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:41.201933 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6301e-7e44-4177-8f27-c2a55d89055f admin admin] PATH_INFO: `/v3/users/ee68fcf9434541e1a3e8aa7d1c0ab57c` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:41.203182 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff6301e-7e44-4177-8f27-c2a55d89055f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee68fcf9434541e1a3e8aa7d1c0ab57c)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:41.205146 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff6301e-7e44-4177-8f27-c2a55d89055f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:41.261953 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-bff6301e-7e44-4177-8f27-c2a55d89055f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee68fcf9434541e1a3e8aa7d1c0ab57c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:41.269424 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-bff6301e-7e44-4177-8f27-c2a55d89055f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee68fcf9434541e1a3e8aa7d1c0ab57c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:41.281670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-bff6301e-7e44-4177-8f27-c2a55d89055f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee68fcf9434541e1a3e8aa7d1c0ab57c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:41.282309 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-bff6301e-7e44-4177-8f27-c2a55d89055f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee68fcf9434541e1a3e8aa7d1c0ab57c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:41.288974 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-bff6301e-7e44-4177-8f27-c2a55d89055f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee68fcf9434541e1a3e8aa7d1c0ab57c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:41.295314 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1001/2020] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:17:41 2022] DELETE /identity/v3/users/ee68fcf9434541e1a3e8aa7d1c0ab57c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:41.561203 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c62b01-946d-47e3-9928-3fd62c5ba7d1 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:41.562682 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-08c62b01-946d-47e3-9928-3fd62c5ba7d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:41.575428 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c62b01-946d-47e3-9928-3fd62c5ba7d1 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:41.634573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e2db4-9cfa-41b0-b6df-02498aa9dba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:41.635479 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e2db4-9cfa-41b0-b6df-02498aa9dba4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:41.635762 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e2db4-9cfa-41b0-b6df-02498aa9dba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:41.636038 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e2db4-9cfa-41b0-b6df-02498aa9dba4 admin admin] PATH_INFO: `/v3/projects/fc781647cd484c83938e6e279fdc97ea` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:41.637210 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4e2db4-9cfa-41b0-b6df-02498aa9dba4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc781647cd484c83938e6e279fdc97ea)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:41.639093 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4e2db4-9cfa-41b0-b6df-02498aa9dba4 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:41.676750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:41.686758 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-ef4e2db4-9cfa-41b0-b6df-02498aa9dba4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc781647cd484c83938e6e279fdc97ea', 'request_id': 'req-ef4e2db4-9cfa-41b0-b6df-02498aa9dba4'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:41.693478 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-ef4e2db4-9cfa-41b0-b6df-02498aa9dba4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc781647cd484c83938e6e279fdc97ea', 'request_id': 'req-ef4e2db4-9cfa-41b0-b6df-02498aa9dba4'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:41.701553 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-ef4e2db4-9cfa-41b0-b6df-02498aa9dba4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc781647cd484c83938e6e279fdc97ea', 'request_id': 'req-ef4e2db4-9cfa-41b0-b6df-02498aa9dba4'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:41.713403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-ef4e2db4-9cfa-41b0-b6df-02498aa9dba4 admin admin] The token cache is being invalidate because project fc781647cd484c83938e6e279fdc97ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:17:41.713696 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-ef4e2db4-9cfa-41b0-b6df-02498aa9dba4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:41.714346 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-ef4e2db4-9cfa-41b0-b6df-02498aa9dba4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:41.717112 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1020/2021] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:17:41 2022] DELETE /identity/v3/projects/fc781647cd484c83938e6e279fdc97ea => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:49.309630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff6301e-7e44-4177-8f27-c2a55d89055f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:49.311038 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-bff6301e-7e44-4177-8f27-c2a55d89055f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:49.323283 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff6301e-7e44-4177-8f27-c2a55d89055f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:49.360824 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:49.380484 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41816252-4375-4771-a591-2ff0b0066107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:49.381360 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41816252-4375-4771-a591-2ff0b0066107 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:49.381646 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41816252-4375-4771-a591-2ff0b0066107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:49.381923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41816252-4375-4771-a591-2ff0b0066107 admin admin] PATH_INFO: `/v3/users/e38ed216fecf4bbc903f0bfc5945581f` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:49.383196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41816252-4375-4771-a591-2ff0b0066107 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e38ed216fecf4bbc903f0bfc5945581f)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:49.385157 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41816252-4375-4771-a591-2ff0b0066107 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:49.434557 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-41816252-4375-4771-a591-2ff0b0066107 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e38ed216fecf4bbc903f0bfc5945581f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:49.442474 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-41816252-4375-4771-a591-2ff0b0066107 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e38ed216fecf4bbc903f0bfc5945581f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:49.454616 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-41816252-4375-4771-a591-2ff0b0066107 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e38ed216fecf4bbc903f0bfc5945581f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:49.455272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-41816252-4375-4771-a591-2ff0b0066107 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e38ed216fecf4bbc903f0bfc5945581f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:49.462368 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-41816252-4375-4771-a591-2ff0b0066107 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e38ed216fecf4bbc903f0bfc5945581f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:49.468875 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1002/2022] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:17:49 2022] DELETE /identity/v3/users/e38ed216fecf4bbc903f0bfc5945581f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:49.473336 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229bb1e-139f-4d51-a44d-becc78c20c6f None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:49.473793 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229bb1e-139f-4d51-a44d-becc78c20c6f None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:49.474034 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229bb1e-139f-4d51-a44d-becc78c20c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:49.492438 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-d229bb1e-139f-4d51-a44d-becc78c20c6f None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:49.493708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-d229bb1e-139f-4d51-a44d-becc78c20c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:49.552394 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1021/2023] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:17:49.562462 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41816252-4375-4771-a591-2ff0b0066107 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:49.563889 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-41816252-4375-4771-a591-2ff0b0066107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:49.618322 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41816252-4375-4771-a591-2ff0b0066107 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:49.621908 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec720207-ac12-4e10-85f5-5ecb050c7ea5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:49.622539 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec720207-ac12-4e10-85f5-5ecb050c7ea5 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:49.622824 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec720207-ac12-4e10-85f5-5ecb050c7ea5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:49.623103 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec720207-ac12-4e10-85f5-5ecb050c7ea5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:49.631802 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec720207-ac12-4e10-85f5-5ecb050c7ea5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:49.633210 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec720207-ac12-4e10-85f5-5ecb050c7ea5 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:49.642373 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1003/2024] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:17:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:17:49.907902 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229bb1e-139f-4d51-a44d-becc78c20c6f None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:49.909270 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-d229bb1e-139f-4d51-a44d-becc78c20c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:49.921226 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229bb1e-139f-4d51-a44d-becc78c20c6f None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:49.926021 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbbc3b0-0c4e-4470-b40f-24a91876d533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:49.927034 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbbc3b0-0c4e-4470-b40f-24a91876d533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:49.927344 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbbc3b0-0c4e-4470-b40f-24a91876d533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:49.927649 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbbc3b0-0c4e-4470-b40f-24a91876d533 admin admin] PATH_INFO: `/v3/projects/164dfe502ebf4e63ae6bc97e0def2de9` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:49.929474 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbbc3b0-0c4e-4470-b40f-24a91876d533 admin admin] RBAC: Authorizing `identity:delete_project(project_id=164dfe502ebf4e63ae6bc97e0def2de9)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:49.931489 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbbc3b0-0c4e-4470-b40f-24a91876d533 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:49.977036 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-7bbbc3b0-0c4e-4470-b40f-24a91876d533 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '164dfe502ebf4e63ae6bc97e0def2de9', 'request_id': 'req-7bbbc3b0-0c4e-4470-b40f-24a91876d533'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:49.983585 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-7bbbc3b0-0c4e-4470-b40f-24a91876d533 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '164dfe502ebf4e63ae6bc97e0def2de9', 'request_id': 'req-7bbbc3b0-0c4e-4470-b40f-24a91876d533'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:49.991452 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-7bbbc3b0-0c4e-4470-b40f-24a91876d533 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '164dfe502ebf4e63ae6bc97e0def2de9', 'request_id': 'req-7bbbc3b0-0c4e-4470-b40f-24a91876d533'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:50.002851 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-7bbbc3b0-0c4e-4470-b40f-24a91876d533 admin admin] The token cache is being invalidate because project 164dfe502ebf4e63ae6bc97e0def2de9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:17:50.003144 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-7bbbc3b0-0c4e-4470-b40f-24a91876d533 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:50.003806 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-7bbbc3b0-0c4e-4470-b40f-24a91876d533 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:17:50.005331 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1022/2025] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:17:49 2022] DELETE /identity/v3/projects/164dfe502ebf4e63ae6bc97e0def2de9 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:50.010492 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721c77b-a3b3-4da1-90ef-d44e7ef680c6 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:50.010856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721c77b-a3b3-4da1-90ef-d44e7ef680c6 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:50.011212 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721c77b-a3b3-4da1-90ef-d44e7ef680c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:50.029390 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-2721c77b-a3b3-4da1-90ef-d44e7ef680c6 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:50.030665 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2721c77b-a3b3-4da1-90ef-d44e7ef680c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:50.092321 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1004/2026] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:17:50.100034 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78588197-086a-43fd-a80c-9da94704a59d None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:50.100326 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78588197-086a-43fd-a80c-9da94704a59d None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:50.100605 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78588197-086a-43fd-a80c-9da94704a59d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:50.120090 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-78588197-086a-43fd-a80c-9da94704a59d None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:50.121412 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-78588197-086a-43fd-a80c-9da94704a59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:50.130001 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1023/2027] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:17:50.134034 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2721c77b-a3b3-4da1-90ef-d44e7ef680c6 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:50.139923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:50.145602 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2721c77b-a3b3-4da1-90ef-d44e7ef680c6 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:50.152093 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f0af52-b7ab-4b90-867c-b0de1580459d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:50.152967 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0af52-b7ab-4b90-867c-b0de1580459d admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:50.153608 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0af52-b7ab-4b90-867c-b0de1580459d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:50.153895 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0af52-b7ab-4b90-867c-b0de1580459d admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:50.154650 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f0af52-b7ab-4b90-867c-b0de1580459d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:17:50.154962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f0af52-b7ab-4b90-867c-b0de1580459d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:50.156636 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f0af52-b7ab-4b90-867c-b0de1580459d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:50.169216 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1005/2028] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:17:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:17:50.173460 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78588197-086a-43fd-a80c-9da94704a59d None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:50.182666 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78588197-086a-43fd-a80c-9da94704a59d None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:50.187559 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d8748-e86e-44e3-9dab-5b8dd22bc3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:50.188223 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d8748-e86e-44e3-9dab-5b8dd22bc3d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:50.188509 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d8748-e86e-44e3-9dab-5b8dd22bc3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:50.188787 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d8748-e86e-44e3-9dab-5b8dd22bc3d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:50.189510 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69d8748-e86e-44e3-9dab-5b8dd22bc3d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:50.191837 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69d8748-e86e-44e3-9dab-5b8dd22bc3d6 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:50.221397 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1024/2029] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:17:50 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:17:50.225595 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f0af52-b7ab-4b90-867c-b0de1580459d admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:50.236240 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f0af52-b7ab-4b90-867c-b0de1580459d admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:50.315465 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f349606d-4c4c-4144-a385-ff5efad9a9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:50.316150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349606d-4c4c-4144-a385-ff5efad9a9f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:50.316425 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349606d-4c4c-4144-a385-ff5efad9a9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:50.316707 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349606d-4c4c-4144-a385-ff5efad9a9f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:50.317423 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f349606d-4c4c-4144-a385-ff5efad9a9f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:50.319379 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f349606d-4c4c-4144-a385-ff5efad9a9f2 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:50.341522 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1006/2030] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:17:50 2022] POST /identity/v3/users => generated 287 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:17:50.345787 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d8748-e86e-44e3-9dab-5b8dd22bc3d6 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:50.356394 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d8748-e86e-44e3-9dab-5b8dd22bc3d6 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:50.361012 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fddb46-38b5-4c87-b9de-736f5c73efee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:50.361673 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fddb46-38b5-4c87-b9de-736f5c73efee admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:50.361962 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fddb46-38b5-4c87-b9de-736f5c73efee admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:50.362236 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fddb46-38b5-4c87-b9de-736f5c73efee admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:50.363410 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fddb46-38b5-4c87-b9de-736f5c73efee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:50.364863 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fddb46-38b5-4c87-b9de-736f5c73efee admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:50.376626 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1025/2031] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:17:50.380503 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f349606d-4c4c-4144-a385-ff5efad9a9f2 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:50.391790 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f349606d-4c4c-4144-a385-ff5efad9a9f2 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:50.396439 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0377b4de-10d0-4296-b14c-3d4ec21dbc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:50.397170 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377b4de-10d0-4296-b14c-3d4ec21dbc8c admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:50.397460 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377b4de-10d0-4296-b14c-3d4ec21dbc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:50.397738 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377b4de-10d0-4296-b14c-3d4ec21dbc8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:50.398473 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0377b4de-10d0-4296-b14c-3d4ec21dbc8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:50.400120 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0377b4de-10d0-4296-b14c-3d4ec21dbc8c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:50.411990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1007/2032] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:17:50.417253 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fddb46-38b5-4c87-b9de-736f5c73efee admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:50.428365 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fddb46-38b5-4c87-b9de-736f5c73efee admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:50.432959 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe0691a-3f62-4d66-9d44-67b0d209909c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:50.433966 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0691a-3f62-4d66-9d44-67b0d209909c admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:50.434248 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0691a-3f62-4d66-9d44-67b0d209909c admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:50.434532 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0691a-3f62-4d66-9d44-67b0d209909c admin admin] PATH_INFO: `/v3/projects/09848a8d962b47439b233d1f06bf72e3/users/334e442aa8d348beaa8dde1f78f5c68d/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:50.437583 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:17:50.452239 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe0691a-3f62-4d66-9d44-67b0d209909c admin admin] RBAC: Authorizing `identity:create_grant(project_id=09848a8d962b47439b233d1f06bf72e3, user_id=334e442aa8d348beaa8dde1f78f5c68d, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:50.454890 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe0691a-3f62-4d66-9d44-67b0d209909c admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:50.468647 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1026/2033] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:17:50 2022] PUT /identity/v3/projects/09848a8d962b47439b233d1f06bf72e3/users/334e442aa8d348beaa8dde1f78f5c68d/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:17:50.478978 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f63d0a-d2a7-4110-ba36-e89801680701 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:50.479272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f63d0a-d2a7-4110-ba36-e89801680701 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:50.479595 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f63d0a-d2a7-4110-ba36-e89801680701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:50.515796 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-e8f63d0a-d2a7-4110-ba36-e89801680701 None None] MFA Rules not processed for user `334e442aa8d348beaa8dde1f78f5c68d`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:50.517094 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e8f63d0a-d2a7-4110-ba36-e89801680701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:50.583349 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1008/2034] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:17:50.594380 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe0691a-3f62-4d66-9d44-67b0d209909c admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:17:50.596091 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-8fe0691a-3f62-4d66-9d44-67b0d209909c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:50.654249 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe0691a-3f62-4d66-9d44-67b0d209909c admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:17:50.658416 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd26d60-3e8e-4fea-9f51-fddc11e0e5f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:17:50.659219 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd26d60-3e8e-4fea-9f51-fddc11e0e5f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:50.659570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd26d60-3e8e-4fea-9f51-fddc11e0e5f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:50.659968 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd26d60-3e8e-4fea-9f51-fddc11e0e5f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:50.669054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd26d60-3e8e-4fea-9f51-fddc11e0e5f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:17:50.670485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd26d60-3e8e-4fea-9f51-fddc11e0e5f2 service cinder] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:17:50.679891 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1027/2035] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:17:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:17:55.228333 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b47321-f72d-44f3-8892-7d596e6d9f99 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:17:55.228333 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b47321-f72d-44f3-8892-7d596e6d9f99 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:17:55.228333 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b47321-f72d-44f3-8892-7d596e6d9f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:17:55.246409 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c3b47321-f72d-44f3-8892-7d596e6d9f99 None None] MFA Rules not processed for user `334e442aa8d348beaa8dde1f78f5c68d`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:17:55.248210 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c3b47321-f72d-44f3-8892-7d596e6d9f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:17:55.256088 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1009/2036] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:17:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:00.057565 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404e6eb-58d0-4901-9002-bcebfcea9b1c None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:00.058017 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404e6eb-58d0-4901-9002-bcebfcea9b1c None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:00.058162 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404e6eb-58d0-4901-9002-bcebfcea9b1c None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:00.059599 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1028/2037] 10.222.0.41 () {54 vars in 921 bytes} [Fri Jun 3 16:18:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:18:00.098637 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee275e6c-77d6-453e-bd6b-f9bd2f00f32a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:00.099137 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee275e6c-77d6-453e-bd6b-f9bd2f00f32a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:00.099360 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee275e6c-77d6-453e-bd6b-f9bd2f00f32a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:00.129448 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-ee275e6c-77d6-453e-bd6b-f9bd2f00f32a None None] MFA Rules not processed for user `be901786f8934d559e0bac5d622f97cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:00.130670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ee275e6c-77d6-453e-bd6b-f9bd2f00f32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:00.133231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404e6eb-58d0-4901-9002-bcebfcea9b1c None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:00.135112 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-6404e6eb-58d0-4901-9002-bcebfcea9b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:00.185809 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:00.281180 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404e6eb-58d0-4901-9002-bcebfcea9b1c None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:00.287231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a929383-a33d-4055-bf06-c3084ee69867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:00.288282 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a929383-a33d-4055-bf06-c3084ee69867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:00.288570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a929383-a33d-4055-bf06-c3084ee69867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:00.288855 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a929383-a33d-4055-bf06-c3084ee69867 admin admin] PATH_INFO: `/v3/users/34ca0974e6dd49bdb6a56ec577dad3cb` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:00.290129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a929383-a33d-4055-bf06-c3084ee69867 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34ca0974e6dd49bdb6a56ec577dad3cb)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:00.292624 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a929383-a33d-4055-bf06-c3084ee69867 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:00.295102 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1010/2038] 10.222.0.41 () {58 vars in 1018 bytes} [Fri Jun 3 16:18:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:00.296630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee275e6c-77d6-453e-bd6b-f9bd2f00f32a None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:00.297539 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ee275e6c-77d6-453e-bd6b-f9bd2f00f32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:00.321686 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee275e6c-77d6-453e-bd6b-f9bd2f00f32a None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:00.336106 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0f316-2ac1-4468-8079-6e0d4e3980bf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be901786f8934d559e0bac5d622f97cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:00.339888 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0f316-2ac1-4468-8079-6e0d4e3980bf service placement] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:00.340260 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0f316-2ac1-4468-8079-6e0d4e3980bf service placement] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:00.340628 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0f316-2ac1-4468-8079-6e0d4e3980bf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:00.353645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0f316-2ac1-4468-8079-6e0d4e3980bf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:00.355506 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0f316-2ac1-4468-8079-6e0d4e3980bf service placement] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:00.363793 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-6a929383-a33d-4055-bf06-c3084ee69867 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34ca0974e6dd49bdb6a56ec577dad3cb'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:00.366017 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:00.371212 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1011/2039] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:18:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:18:00.381363 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-6a929383-a33d-4055-bf06-c3084ee69867 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34ca0974e6dd49bdb6a56ec577dad3cb'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:00.397397 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-6a929383-a33d-4055-bf06-c3084ee69867 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34ca0974e6dd49bdb6a56ec577dad3cb'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:00.410275 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-6a929383-a33d-4055-bf06-c3084ee69867 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34ca0974e6dd49bdb6a56ec577dad3cb'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:00.411246 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-6a929383-a33d-4055-bf06-c3084ee69867 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34ca0974e6dd49bdb6a56ec577dad3cb'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:00.426168 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1029/2040] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:18:00 2022] DELETE /identity/v3/users/34ca0974e6dd49bdb6a56ec577dad3cb => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:18:00.874908 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0f316-2ac1-4468-8079-6e0d4e3980bf service placement] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:00.885664 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0f316-2ac1-4468-8079-6e0d4e3980bf service placement] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:00.954265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f0a2d1-648c-47a6-9977-9e561829b4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:00.955129 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f0a2d1-648c-47a6-9977-9e561829b4cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:00.955460 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f0a2d1-648c-47a6-9977-9e561829b4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:00.955738 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f0a2d1-648c-47a6-9977-9e561829b4cc admin admin] PATH_INFO: `/v3/projects/b1dde95632be44e29e77bfddc2ad84b5` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:00.956921 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f0a2d1-648c-47a6-9977-9e561829b4cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1dde95632be44e29e77bfddc2ad84b5)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:00.958815 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f0a2d1-648c-47a6-9977-9e561829b4cc admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:01.007728 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c2f0a2d1-648c-47a6-9977-9e561829b4cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1dde95632be44e29e77bfddc2ad84b5', 'request_id': 'req-c2f0a2d1-648c-47a6-9977-9e561829b4cc'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:01.014338 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c2f0a2d1-648c-47a6-9977-9e561829b4cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1dde95632be44e29e77bfddc2ad84b5', 'request_id': 'req-c2f0a2d1-648c-47a6-9977-9e561829b4cc'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:01.021391 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c2f0a2d1-648c-47a6-9977-9e561829b4cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1dde95632be44e29e77bfddc2ad84b5', 'request_id': 'req-c2f0a2d1-648c-47a6-9977-9e561829b4cc'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:01.031905 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c2f0a2d1-648c-47a6-9977-9e561829b4cc admin admin] The token cache is being invalidate because project b1dde95632be44e29e77bfddc2ad84b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:18:01.032191 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c2f0a2d1-648c-47a6-9977-9e561829b4cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:01.032808 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c2f0a2d1-648c-47a6-9977-9e561829b4cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:01.034332 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1012/2041] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:18:00 2022] DELETE /identity/v3/projects/b1dde95632be44e29e77bfddc2ad84b5 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:18:01.039263 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f589527-babe-41de-9fb3-01c568486a47 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:01.039591 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f589527-babe-41de-9fb3-01c568486a47 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:01.039876 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f589527-babe-41de-9fb3-01c568486a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:01.087952 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-6f589527-babe-41de-9fb3-01c568486a47 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:01.089184 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-6f589527-babe-41de-9fb3-01c568486a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:01.152411 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1030/2042] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:01.160463 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f730a6-a012-46bc-9071-10c57bd8f43a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:01.161100 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f730a6-a012-46bc-9071-10c57bd8f43a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:01.161100 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f730a6-a012-46bc-9071-10c57bd8f43a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:01.179149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-03f730a6-a012-46bc-9071-10c57bd8f43a None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:01.180383 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-03f730a6-a012-46bc-9071-10c57bd8f43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:01.188689 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1013/2043] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:01.192879 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f589527-babe-41de-9fb3-01c568486a47 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:01.203610 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f589527-babe-41de-9fb3-01c568486a47 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:01.208186 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38b54dc-3cb0-46d2-89e6-33c93e5ccb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:01.208851 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b54dc-3cb0-46d2-89e6-33c93e5ccb35 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:01.209134 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b54dc-3cb0-46d2-89e6-33c93e5ccb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:01.209410 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b54dc-3cb0-46d2-89e6-33c93e5ccb35 admin admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:01.210139 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38b54dc-3cb0-46d2-89e6-33c93e5ccb35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:18:01.210460 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38b54dc-3cb0-46d2-89e6-33c93e5ccb35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:01.212094 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38b54dc-3cb0-46d2-89e6-33c93e5ccb35 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:01.224045 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1031/2044] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:18:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:18:01.228433 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f730a6-a012-46bc-9071-10c57bd8f43a None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:01.240090 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f730a6-a012-46bc-9071-10c57bd8f43a None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:01.245161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4172a0-0789-41c9-8f61-e0213ec465bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:01.245945 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4172a0-0789-41c9-8f61-e0213ec465bb admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:01.246231 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4172a0-0789-41c9-8f61-e0213ec465bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:01.246514 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4172a0-0789-41c9-8f61-e0213ec465bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:01.247226 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4172a0-0789-41c9-8f61-e0213ec465bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:01.249179 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4172a0-0789-41c9-8f61-e0213ec465bb admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:01.253720 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:01.277616 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1014/2045] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:18:01 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:18:01.281540 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38b54dc-3cb0-46d2-89e6-33c93e5ccb35 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:01.291813 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38b54dc-3cb0-46d2-89e6-33c93e5ccb35 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:01.347413 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886128c-5ac3-4087-b79b-30702be06be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:01.348086 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886128c-5ac3-4087-b79b-30702be06be2 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:01.348373 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886128c-5ac3-4087-b79b-30702be06be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:01.348651 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886128c-5ac3-4087-b79b-30702be06be2 admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:01.349345 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f886128c-5ac3-4087-b79b-30702be06be2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:01.351243 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f886128c-5ac3-4087-b79b-30702be06be2 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:01.373439 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1032/2046] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:18:01 2022] POST /identity/v3/users => generated 284 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:18:01.377498 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4172a0-0789-41c9-8f61-e0213ec465bb admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:01.388139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4172a0-0789-41c9-8f61-e0213ec465bb admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:01.392718 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98af8449-e397-440a-88c2-b25079ce44fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:01.393380 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af8449-e397-440a-88c2-b25079ce44fe admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:01.393665 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af8449-e397-440a-88c2-b25079ce44fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:01.393943 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af8449-e397-440a-88c2-b25079ce44fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:01.394666 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98af8449-e397-440a-88c2-b25079ce44fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:01.396294 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98af8449-e397-440a-88c2-b25079ce44fe admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:01.407105 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1015/2047] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:18:01.410739 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886128c-5ac3-4087-b79b-30702be06be2 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:01.420610 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886128c-5ac3-4087-b79b-30702be06be2 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:01.425259 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2232d7-07e4-42b7-8184-ca0115bfa51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:01.425911 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2232d7-07e4-42b7-8184-ca0115bfa51a admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:01.426196 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2232d7-07e4-42b7-8184-ca0115bfa51a admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:01.426499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2232d7-07e4-42b7-8184-ca0115bfa51a admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:01.427230 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2232d7-07e4-42b7-8184-ca0115bfa51a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:01.429191 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2232d7-07e4-42b7-8184-ca0115bfa51a admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:01.439010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1033/2048] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:18:01.442965 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98af8449-e397-440a-88c2-b25079ce44fe admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:01.453193 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98af8449-e397-440a-88c2-b25079ce44fe admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:01.457686 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4efaea2-6a49-4cbf-a067-69d041de048e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:01.458561 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efaea2-6a49-4cbf-a067-69d041de048e admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:01.458849 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efaea2-6a49-4cbf-a067-69d041de048e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:01.459129 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efaea2-6a49-4cbf-a067-69d041de048e admin admin] PATH_INFO: `/v3/projects/9f24b497164f45348ac2f7ec664c6555/users/ac767819c00f42618817e60da5b3eab2/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:01.475231 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4efaea2-6a49-4cbf-a067-69d041de048e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f24b497164f45348ac2f7ec664c6555, user_id=ac767819c00f42618817e60da5b3eab2, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:01.477903 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4efaea2-6a49-4cbf-a067-69d041de048e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:01.491949 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1016/2049] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:18:01 2022] PUT /identity/v3/projects/9f24b497164f45348ac2f7ec664c6555/users/ac767819c00f42618817e60da5b3eab2/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:18:01.502180 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bc991-1ffb-4e2e-971b-238c6920a66a None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:01.502675 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bc991-1ffb-4e2e-971b-238c6920a66a None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:01.503076 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bc991-1ffb-4e2e-971b-238c6920a66a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:01.522274 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:01.534421 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-b14bc991-1ffb-4e2e-971b-238c6920a66a None None] MFA Rules not processed for user `ac767819c00f42618817e60da5b3eab2`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:01.535717 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-b14bc991-1ffb-4e2e-971b-238c6920a66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:01.605252 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1034/2050] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:01 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:18:01.610052 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4efaea2-6a49-4cbf-a067-69d041de048e admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:01.682822 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4efaea2-6a49-4cbf-a067-69d041de048e admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:01.688111 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5a677e-2ba4-484a-a970-86e040595433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:01.688765 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5a677e-2ba4-484a-a970-86e040595433 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:01.689050 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5a677e-2ba4-484a-a970-86e040595433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:01.689330 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5a677e-2ba4-484a-a970-86e040595433 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:01.690012 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5a677e-2ba4-484a-a970-86e040595433 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:01.691960 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5a677e-2ba4-484a-a970-86e040595433 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:01.716230 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1017/2051] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:18:01 2022] POST /identity/v3/projects => generated 367 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:18:01.720160 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14bc991-1ffb-4e2e-971b-238c6920a66a None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:01.731099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14bc991-1ffb-4e2e-971b-238c6920a66a None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:01.792853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed188f-8d57-483f-9d66-572d622778a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:01.793517 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed188f-8d57-483f-9d66-572d622778a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:01.793798 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed188f-8d57-483f-9d66-572d622778a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:01.794073 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed188f-8d57-483f-9d66-572d622778a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:01.794762 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed188f-8d57-483f-9d66-572d622778a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:01.796691 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed188f-8d57-483f-9d66-572d622778a3 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:01.817338 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1035/2052] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:18:01 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:18:01.821308 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5a677e-2ba4-484a-a970-86e040595433 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:01.832393 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5a677e-2ba4-484a-a970-86e040595433 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:01.837375 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5da6f0-be6f-4d48-9803-10374677a504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:01.838052 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5da6f0-be6f-4d48-9803-10374677a504 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:01.838377 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5da6f0-be6f-4d48-9803-10374677a504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:01.838596 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5da6f0-be6f-4d48-9803-10374677a504 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:01.839382 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5da6f0-be6f-4d48-9803-10374677a504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:01.840974 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5da6f0-be6f-4d48-9803-10374677a504 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:01.851877 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1018/2053] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:18:01.856126 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed188f-8d57-483f-9d66-572d622778a3 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:01.865955 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed188f-8d57-483f-9d66-572d622778a3 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:01.871081 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbee278-834c-486e-a2e1-65e720d2e75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:01.872061 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbee278-834c-486e-a2e1-65e720d2e75f admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:01.872354 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbee278-834c-486e-a2e1-65e720d2e75f admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:01.872633 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbee278-834c-486e-a2e1-65e720d2e75f admin admin] PATH_INFO: `/v3/projects/a577dcd0f9914e8a910ca3bae10a376b/users/27fdfeedc5884f909f97df5409601e0e/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:01.891104 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbee278-834c-486e-a2e1-65e720d2e75f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a577dcd0f9914e8a910ca3bae10a376b, user_id=27fdfeedc5884f909f97df5409601e0e, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:01.894144 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbee278-834c-486e-a2e1-65e720d2e75f admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:01.909599 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1036/2054] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:18:01 2022] PUT /identity/v3/projects/a577dcd0f9914e8a910ca3bae10a376b/users/27fdfeedc5884f909f97df5409601e0e/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:18:01.913588 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5da6f0-be6f-4d48-9803-10374677a504 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:01.924528 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5da6f0-be6f-4d48-9803-10374677a504 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:01.982150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e058d-f155-47c5-abf4-5393e53945c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:01.982821 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e058d-f155-47c5-abf4-5393e53945c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:01.983105 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e058d-f155-47c5-abf4-5393e53945c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:01.983406 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e058d-f155-47c5-abf4-5393e53945c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:01.984134 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018e058d-f155-47c5-abf4-5393e53945c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:01.985751 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018e058d-f155-47c5-abf4-5393e53945c5 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:01.995526 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1019/2055] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:18:02.000410 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbee278-834c-486e-a2e1-65e720d2e75f admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:02.010352 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbee278-834c-486e-a2e1-65e720d2e75f admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:02.015678 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71429404-1c92-4035-914c-8b8fd4c28fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:02.016566 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71429404-1c92-4035-914c-8b8fd4c28fc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:02.016877 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71429404-1c92-4035-914c-8b8fd4c28fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:02.017306 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71429404-1c92-4035-914c-8b8fd4c28fc7 admin admin] PATH_INFO: `/v3/projects/a577dcd0f9914e8a910ca3bae10a376b/users/27fdfeedc5884f909f97df5409601e0e/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:02.019648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71429404-1c92-4035-914c-8b8fd4c28fc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a577dcd0f9914e8a910ca3bae10a376b, user_id=27fdfeedc5884f909f97df5409601e0e, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:02.022856 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71429404-1c92-4035-914c-8b8fd4c28fc7 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:02.036256 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1037/2056] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:18:01 2022] PUT /identity/v3/projects/a577dcd0f9914e8a910ca3bae10a376b/users/27fdfeedc5884f909f97df5409601e0e/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:18:02.047790 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe69ffa-20c6-451a-8c09-c8d0d3a38539 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:02.048096 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe69ffa-20c6-451a-8c09-c8d0d3a38539 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:02.048383 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe69ffa-20c6-451a-8c09-c8d0d3a38539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:02.075723 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-dbe69ffa-20c6-451a-8c09-c8d0d3a38539 None None] MFA Rules not processed for user `27fdfeedc5884f909f97df5409601e0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:02.077112 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-dbe69ffa-20c6-451a-8c09-c8d0d3a38539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:02.086045 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:02.144375 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1020/2057] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:02 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:02.156079 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71429404-1c92-4035-914c-8b8fd4c28fc7 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:02.157914 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-71429404-1c92-4035-914c-8b8fd4c28fc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:02.213861 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71429404-1c92-4035-914c-8b8fd4c28fc7 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:02.218279 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e5a1b-6b56-4a96-9193-e848af826558 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:02.218939 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e5a1b-6b56-4a96-9193-e848af826558 service cinder] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:02.219225 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e5a1b-6b56-4a96-9193-e848af826558 service cinder] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:02.219587 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e5a1b-6b56-4a96-9193-e848af826558 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:02.226193 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:02.232893 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1e5a1b-6b56-4a96-9193-e848af826558 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:02.234337 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1e5a1b-6b56-4a96-9193-e848af826558 service cinder] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:02.244609 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1038/2058] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:18:02 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:18:03.635283 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe69ffa-20c6-451a-8c09-c8d0d3a38539 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:03.699343 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe69ffa-20c6-451a-8c09-c8d0d3a38539 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:03.703734 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab324367-c7c3-417d-909c-6847adc3f23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:03.704598 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab324367-c7c3-417d-909c-6847adc3f23c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:03.704889 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab324367-c7c3-417d-909c-6847adc3f23c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:03.705159 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab324367-c7c3-417d-909c-6847adc3f23c admin admin] PATH_INFO: `/v3/users/ac767819c00f42618817e60da5b3eab2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:03.706338 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab324367-c7c3-417d-909c-6847adc3f23c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac767819c00f42618817e60da5b3eab2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:03.708320 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab324367-c7c3-417d-909c-6847adc3f23c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:03.755748 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ab324367-c7c3-417d-909c-6847adc3f23c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac767819c00f42618817e60da5b3eab2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:03.764419 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ab324367-c7c3-417d-909c-6847adc3f23c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac767819c00f42618817e60da5b3eab2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:03.776622 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ab324367-c7c3-417d-909c-6847adc3f23c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac767819c00f42618817e60da5b3eab2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:03.777801 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ab324367-c7c3-417d-909c-6847adc3f23c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac767819c00f42618817e60da5b3eab2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:03.784503 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ab324367-c7c3-417d-909c-6847adc3f23c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac767819c00f42618817e60da5b3eab2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:03.790446 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1021/2059] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:18:03 2022] DELETE /identity/v3/users/ac767819c00f42618817e60da5b3eab2 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:18:03.794855 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c7593-24e7-47f0-b4c0-864b562d30e3 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:03.795329 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c7593-24e7-47f0-b4c0-864b562d30e3 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:03.795475 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c7593-24e7-47f0-b4c0-864b562d30e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:03.812734 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-f92c7593-24e7-47f0-b4c0-864b562d30e3 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:03.813937 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-f92c7593-24e7-47f0-b4c0-864b562d30e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:03.887743 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1039/2060] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:03.890029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab324367-c7c3-417d-909c-6847adc3f23c admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:03.891558 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ab324367-c7c3-417d-909c-6847adc3f23c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:03.924525 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:03.978889 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab324367-c7c3-417d-909c-6847adc3f23c admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:03.983555 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cbd89c-471a-44e1-b79e-bde60e3114ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:03.984210 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cbd89c-471a-44e1-b79e-bde60e3114ea service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:03.984495 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cbd89c-471a-44e1-b79e-bde60e3114ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:03.984773 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cbd89c-471a-44e1-b79e-bde60e3114ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:03.994440 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cbd89c-471a-44e1-b79e-bde60e3114ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:03.996607 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cbd89c-471a-44e1-b79e-bde60e3114ea service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:04.006339 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1022/2061] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:18:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:18:04.279652 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c7593-24e7-47f0-b4c0-864b562d30e3 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:04.281158 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-f92c7593-24e7-47f0-b4c0-864b562d30e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:04.300541 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c7593-24e7-47f0-b4c0-864b562d30e3 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:04.306153 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b57ecfb-c399-4e31-a26e-aa2bd7752428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:04.307124 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57ecfb-c399-4e31-a26e-aa2bd7752428 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:04.308709 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57ecfb-c399-4e31-a26e-aa2bd7752428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:04.308709 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57ecfb-c399-4e31-a26e-aa2bd7752428 admin admin] PATH_INFO: `/v3/projects/9f24b497164f45348ac2f7ec664c6555` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:04.309891 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b57ecfb-c399-4e31-a26e-aa2bd7752428 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f24b497164f45348ac2f7ec664c6555)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:04.312116 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b57ecfb-c399-4e31-a26e-aa2bd7752428 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:04.361268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-6b57ecfb-c399-4e31-a26e-aa2bd7752428 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f24b497164f45348ac2f7ec664c6555', 'request_id': 'req-6b57ecfb-c399-4e31-a26e-aa2bd7752428'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:04.367557 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-6b57ecfb-c399-4e31-a26e-aa2bd7752428 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f24b497164f45348ac2f7ec664c6555', 'request_id': 'req-6b57ecfb-c399-4e31-a26e-aa2bd7752428'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:04.375751 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-6b57ecfb-c399-4e31-a26e-aa2bd7752428 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f24b497164f45348ac2f7ec664c6555', 'request_id': 'req-6b57ecfb-c399-4e31-a26e-aa2bd7752428'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:04.387502 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-6b57ecfb-c399-4e31-a26e-aa2bd7752428 admin admin] The token cache is being invalidate because project 9f24b497164f45348ac2f7ec664c6555 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:18:04.387896 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-6b57ecfb-c399-4e31-a26e-aa2bd7752428 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:04.388658 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-6b57ecfb-c399-4e31-a26e-aa2bd7752428 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:04.390736 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1040/2062] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:18:04 2022] DELETE /identity/v3/projects/9f24b497164f45348ac2f7ec664c6555 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:18:04.394764 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cbd89c-471a-44e1-b79e-bde60e3114ea service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:04.407178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cbd89c-471a-44e1-b79e-bde60e3114ea service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:04.462390 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0aec1c-4f1f-4100-bb06-ffb6ef5323f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:04.463278 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0aec1c-4f1f-4100-bb06-ffb6ef5323f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:04.463581 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0aec1c-4f1f-4100-bb06-ffb6ef5323f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:04.463859 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0aec1c-4f1f-4100-bb06-ffb6ef5323f9 admin admin] PATH_INFO: `/v3/users/27fdfeedc5884f909f97df5409601e0e` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:04.465712 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0aec1c-4f1f-4100-bb06-ffb6ef5323f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27fdfeedc5884f909f97df5409601e0e)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:04.467690 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0aec1c-4f1f-4100-bb06-ffb6ef5323f9 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:04.512198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-cd0aec1c-4f1f-4100-bb06-ffb6ef5323f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27fdfeedc5884f909f97df5409601e0e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:04.519855 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-cd0aec1c-4f1f-4100-bb06-ffb6ef5323f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27fdfeedc5884f909f97df5409601e0e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:04.534138 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-cd0aec1c-4f1f-4100-bb06-ffb6ef5323f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27fdfeedc5884f909f97df5409601e0e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:04.534898 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-cd0aec1c-4f1f-4100-bb06-ffb6ef5323f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27fdfeedc5884f909f97df5409601e0e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:04.541560 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-cd0aec1c-4f1f-4100-bb06-ffb6ef5323f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27fdfeedc5884f909f97df5409601e0e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:04.548005 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1023/2063] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:18:04 2022] DELETE /identity/v3/users/27fdfeedc5884f909f97df5409601e0e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:18:04.811017 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b57ecfb-c399-4e31-a26e-aa2bd7752428 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:04.812523 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-6b57ecfb-c399-4e31-a26e-aa2bd7752428 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:04.826304 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b57ecfb-c399-4e31-a26e-aa2bd7752428 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:04.883162 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe20bd-c66c-43f2-a4bc-b2ceac90d4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:04.884091 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe20bd-c66c-43f2-a4bc-b2ceac90d4f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:04.884376 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe20bd-c66c-43f2-a4bc-b2ceac90d4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:04.884653 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe20bd-c66c-43f2-a4bc-b2ceac90d4f7 admin admin] PATH_INFO: `/v3/projects/a577dcd0f9914e8a910ca3bae10a376b` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:04.885885 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fe20bd-c66c-43f2-a4bc-b2ceac90d4f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a577dcd0f9914e8a910ca3bae10a376b)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:04.887813 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fe20bd-c66c-43f2-a4bc-b2ceac90d4f7 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:04.919741 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:04.945819 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-14fe20bd-c66c-43f2-a4bc-b2ceac90d4f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a577dcd0f9914e8a910ca3bae10a376b', 'request_id': 'req-14fe20bd-c66c-43f2-a4bc-b2ceac90d4f7'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:04.952440 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-14fe20bd-c66c-43f2-a4bc-b2ceac90d4f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a577dcd0f9914e8a910ca3bae10a376b', 'request_id': 'req-14fe20bd-c66c-43f2-a4bc-b2ceac90d4f7'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:04.959517 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-14fe20bd-c66c-43f2-a4bc-b2ceac90d4f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a577dcd0f9914e8a910ca3bae10a376b', 'request_id': 'req-14fe20bd-c66c-43f2-a4bc-b2ceac90d4f7'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:04.969583 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-14fe20bd-c66c-43f2-a4bc-b2ceac90d4f7 admin admin] The token cache is being invalidate because project a577dcd0f9914e8a910ca3bae10a376b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:18:04.969872 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-14fe20bd-c66c-43f2-a4bc-b2ceac90d4f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:04.970486 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-14fe20bd-c66c-43f2-a4bc-b2ceac90d4f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:04.972050 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1041/2064] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:18:04 2022] DELETE /identity/v3/projects/a577dcd0f9914e8a910ca3bae10a376b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:18:04.977026 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a0b23-13d4-40e8-a22f-a1950db4931e None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:04.977482 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a0b23-13d4-40e8-a22f-a1950db4931e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:04.977572 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a0b23-13d4-40e8-a22f-a1950db4931e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:04.994918 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b89a0b23-13d4-40e8-a22f-a1950db4931e None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:04.996011 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b89a0b23-13d4-40e8-a22f-a1950db4931e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:05.020641 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:05.060753 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1024/2065] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:05.068260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98064605-b87f-44f7-838f-ce5329d9356b None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:05.068837 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98064605-b87f-44f7-838f-ce5329d9356b None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:05.069122 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98064605-b87f-44f7-838f-ce5329d9356b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:05.090912 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-98064605-b87f-44f7-838f-ce5329d9356b None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:05.092316 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-98064605-b87f-44f7-838f-ce5329d9356b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:05.100533 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1042/2066] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:05.104946 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a0b23-13d4-40e8-a22f-a1950db4931e None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:05.114511 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a0b23-13d4-40e8-a22f-a1950db4931e None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:05.119082 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc1101-5fcd-4934-a900-03a07364436c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:05.119768 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc1101-5fcd-4934-a900-03a07364436c admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:05.120057 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc1101-5fcd-4934-a900-03a07364436c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:05.120335 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc1101-5fcd-4934-a900-03a07364436c admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:05.121125 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fc1101-5fcd-4934-a900-03a07364436c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:18:05.121436 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fc1101-5fcd-4934-a900-03a07364436c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:05.123034 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fc1101-5fcd-4934-a900-03a07364436c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:05.134844 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1025/2067] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:18:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:18:05.139715 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98064605-b87f-44f7-838f-ce5329d9356b None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:05.149273 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98064605-b87f-44f7-838f-ce5329d9356b None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:05.155119 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d07fd2-22dd-4459-9740-9ea4c00671e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:05.155866 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d07fd2-22dd-4459-9740-9ea4c00671e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:05.156150 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d07fd2-22dd-4459-9740-9ea4c00671e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:05.156426 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d07fd2-22dd-4459-9740-9ea4c00671e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:05.157108 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d07fd2-22dd-4459-9740-9ea4c00671e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:05.158983 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d07fd2-22dd-4459-9740-9ea4c00671e9 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:05.186142 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1043/2068] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:18:05 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:18:05.188275 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc1101-5fcd-4934-a900-03a07364436c admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:05.199087 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc1101-5fcd-4934-a900-03a07364436c admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:05.255485 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34bbf0-d10e-42ce-9691-da299997c4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:05.256155 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34bbf0-d10e-42ce-9691-da299997c4ff admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:05.256468 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34bbf0-d10e-42ce-9691-da299997c4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:05.256747 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34bbf0-d10e-42ce-9691-da299997c4ff admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:05.257451 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d34bbf0-d10e-42ce-9691-da299997c4ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:05.259407 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d34bbf0-d10e-42ce-9691-da299997c4ff admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:05.280169 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1026/2069] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:18:05 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:18:05.284193 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d07fd2-22dd-4459-9740-9ea4c00671e9 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:05.294256 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d07fd2-22dd-4459-9740-9ea4c00671e9 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:05.298749 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e9849-5ddd-49b1-a24d-e8c118a8c7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:05.299431 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e9849-5ddd-49b1-a24d-e8c118a8c7d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:05.299714 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e9849-5ddd-49b1-a24d-e8c118a8c7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:05.299990 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e9849-5ddd-49b1-a24d-e8c118a8c7d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:05.300702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41e9849-5ddd-49b1-a24d-e8c118a8c7d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:05.302270 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41e9849-5ddd-49b1-a24d-e8c118a8c7d3 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:05.312315 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1044/2070] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:18:05.315929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34bbf0-d10e-42ce-9691-da299997c4ff admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:05.326071 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34bbf0-d10e-42ce-9691-da299997c4ff admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:05.331135 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6d4040-79ff-4e4b-86bf-4b1136a02dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:05.331834 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d4040-79ff-4e4b-86bf-4b1136a02dbe admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:05.332120 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d4040-79ff-4e4b-86bf-4b1136a02dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:05.332400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d4040-79ff-4e4b-86bf-4b1136a02dbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:05.333154 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6d4040-79ff-4e4b-86bf-4b1136a02dbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:05.334737 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6d4040-79ff-4e4b-86bf-4b1136a02dbe admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:05.346477 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1027/2071] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:18:05.351575 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e9849-5ddd-49b1-a24d-e8c118a8c7d3 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:05.361457 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e9849-5ddd-49b1-a24d-e8c118a8c7d3 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:05.365965 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eebe9e7-2d90-4543-b0cb-7633ef214de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:05.366829 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eebe9e7-2d90-4543-b0cb-7633ef214de7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:05.367105 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eebe9e7-2d90-4543-b0cb-7633ef214de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:05.367414 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eebe9e7-2d90-4543-b0cb-7633ef214de7 admin admin] PATH_INFO: `/v3/projects/d90dd72a1f1547239fc123a0f149cace/users/5f40166346f948b28986447ab392021c/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:05.382766 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eebe9e7-2d90-4543-b0cb-7633ef214de7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d90dd72a1f1547239fc123a0f149cace, user_id=5f40166346f948b28986447ab392021c, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:05.385721 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eebe9e7-2d90-4543-b0cb-7633ef214de7 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:05.398597 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1045/2072] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:18:05 2022] PUT /identity/v3/projects/d90dd72a1f1547239fc123a0f149cace/users/5f40166346f948b28986447ab392021c/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:18:05.409059 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c492fe5-3d32-46d8-a348-9b6875d4ef64 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:05.409357 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c492fe5-3d32-46d8-a348-9b6875d4ef64 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:05.409646 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c492fe5-3d32-46d8-a348-9b6875d4ef64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:05.436611 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-8c492fe5-3d32-46d8-a348-9b6875d4ef64 None None] MFA Rules not processed for user `5f40166346f948b28986447ab392021c`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:05.437825 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8c492fe5-3d32-46d8-a348-9b6875d4ef64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:05.515355 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1028/2073] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:05.518283 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eebe9e7-2d90-4543-b0cb-7633ef214de7 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:05.596896 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eebe9e7-2d90-4543-b0cb-7633ef214de7 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:05.601632 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c96eb81-7890-4cb1-81cc-a51d804e501b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:05.602286 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96eb81-7890-4cb1-81cc-a51d804e501b admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:05.602568 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96eb81-7890-4cb1-81cc-a51d804e501b admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:05.602845 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96eb81-7890-4cb1-81cc-a51d804e501b admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:05.603591 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c96eb81-7890-4cb1-81cc-a51d804e501b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:05.605465 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c96eb81-7890-4cb1-81cc-a51d804e501b admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:05.639665 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1046/2074] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:18:05 2022] POST /identity/v3/projects => generated 381 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:18:05.643454 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c492fe5-3d32-46d8-a348-9b6875d4ef64 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:05.656537 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c492fe5-3d32-46d8-a348-9b6875d4ef64 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:05.683542 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:05.726450 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f58358-9506-42cb-9e29-20d1b08f7b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:05.727196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f58358-9506-42cb-9e29-20d1b08f7b5b admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:05.727537 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f58358-9506-42cb-9e29-20d1b08f7b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:05.727814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f58358-9506-42cb-9e29-20d1b08f7b5b admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:05.728522 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f58358-9506-42cb-9e29-20d1b08f7b5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:05.731241 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f58358-9506-42cb-9e29-20d1b08f7b5b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:05.759167 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1029/2075] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:18:05 2022] POST /identity/v3/users => generated 297 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:18:05.762991 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c96eb81-7890-4cb1-81cc-a51d804e501b admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:05.774272 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c96eb81-7890-4cb1-81cc-a51d804e501b admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:05.779237 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5170b-18b3-42e3-bced-3a91e2ed43f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:05.780062 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5170b-18b3-42e3-bced-3a91e2ed43f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:05.780345 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5170b-18b3-42e3-bced-3a91e2ed43f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:05.780620 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5170b-18b3-42e3-bced-3a91e2ed43f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:05.781339 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf5170b-18b3-42e3-bced-3a91e2ed43f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:05.782919 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf5170b-18b3-42e3-bced-3a91e2ed43f9 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:05.795182 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1047/2076] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:18:05.799748 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f58358-9506-42cb-9e29-20d1b08f7b5b admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:05.811179 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f58358-9506-42cb-9e29-20d1b08f7b5b admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:05.816153 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8dadd-1c1a-4172-95f5-43b614388d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:05.817047 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8dadd-1c1a-4172-95f5-43b614388d1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:05.817332 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8dadd-1c1a-4172-95f5-43b614388d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:05.817604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8dadd-1c1a-4172-95f5-43b614388d1e admin admin] PATH_INFO: `/v3/projects/b26c390f1a8c448aacd623d400d365ad/users/8653b4121a43429c85af72f699b35c74/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:05.833692 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f8dadd-1c1a-4172-95f5-43b614388d1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b26c390f1a8c448aacd623d400d365ad, user_id=8653b4121a43429c85af72f699b35c74, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:05.836368 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f8dadd-1c1a-4172-95f5-43b614388d1e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:05.848773 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1030/2077] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:18:05 2022] PUT /identity/v3/projects/b26c390f1a8c448aacd623d400d365ad/users/8653b4121a43429c85af72f699b35c74/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:18:05.852577 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5170b-18b3-42e3-bced-3a91e2ed43f9 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:05.863082 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5170b-18b3-42e3-bced-3a91e2ed43f9 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:05.882187 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:05.928870 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebb0f75-22c0-4123-96bb-2b77c3bcb234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:05.929605 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb0f75-22c0-4123-96bb-2b77c3bcb234 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:05.929889 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb0f75-22c0-4123-96bb-2b77c3bcb234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:05.930164 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb0f75-22c0-4123-96bb-2b77c3bcb234 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:05.930887 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebb0f75-22c0-4123-96bb-2b77c3bcb234 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:05.932509 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebb0f75-22c0-4123-96bb-2b77c3bcb234 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:05.944009 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1048/2078] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:18:05.948163 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8dadd-1c1a-4172-95f5-43b614388d1e admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:05.958241 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8dadd-1c1a-4172-95f5-43b614388d1e admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:05.963219 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d683d-f98c-4283-9c6e-5fd6314894f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:05.964178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d683d-f98c-4283-9c6e-5fd6314894f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:05.964463 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d683d-f98c-4283-9c6e-5fd6314894f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:05.964737 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d683d-f98c-4283-9c6e-5fd6314894f3 admin admin] PATH_INFO: `/v3/projects/b26c390f1a8c448aacd623d400d365ad/users/8653b4121a43429c85af72f699b35c74/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:05.966740 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d683d-f98c-4283-9c6e-5fd6314894f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b26c390f1a8c448aacd623d400d365ad, user_id=8653b4121a43429c85af72f699b35c74, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:05.969999 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d683d-f98c-4283-9c6e-5fd6314894f3 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:05.999863 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1031/2079] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:18:05 2022] PUT /identity/v3/projects/b26c390f1a8c448aacd623d400d365ad/users/8653b4121a43429c85af72f699b35c74/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:18:06.013103 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a750b3f2-28c4-42f9-b289-f49c82cd8a8f None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:06.013471 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a750b3f2-28c4-42f9-b289-f49c82cd8a8f None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:06.013826 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a750b3f2-28c4-42f9-b289-f49c82cd8a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:06.041580 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-a750b3f2-28c4-42f9-b289-f49c82cd8a8f None None] MFA Rules not processed for user `8653b4121a43429c85af72f699b35c74`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:06.043465 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-a750b3f2-28c4-42f9-b289-f49c82cd8a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:06.116304 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1049/2080] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:06 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:06.129671 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d683d-f98c-4283-9c6e-5fd6314894f3 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:06.131420 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-848d683d-f98c-4283-9c6e-5fd6314894f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:06.191025 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d683d-f98c-4283-9c6e-5fd6314894f3 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:06.194852 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa19218-a2ed-43cd-ad52-4f63f63e364d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:06.195532 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa19218-a2ed-43cd-ad52-4f63f63e364d service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:06.195824 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa19218-a2ed-43cd-ad52-4f63f63e364d service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:06.196105 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa19218-a2ed-43cd-ad52-4f63f63e364d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:06.205282 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa19218-a2ed-43cd-ad52-4f63f63e364d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:06.206721 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa19218-a2ed-43cd-ad52-4f63f63e364d service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:06.216956 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1032/2081] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:18:06 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:18:06.529678 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a750b3f2-28c4-42f9-b289-f49c82cd8a8f None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:06.538813 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a750b3f2-28c4-42f9-b289-f49c82cd8a8f None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:06.543247 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868a9dc-c774-4ad5-bda9-916ac5a127df tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8653b4121a43429c85af72f699b35c74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b26c390f1a8c448aacd623d400d365ad', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:06.543927 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868a9dc-c774-4ad5-bda9-916ac5a127df tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:06.544214 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868a9dc-c774-4ad5-bda9-916ac5a127df tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:06.544492 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868a9dc-c774-4ad5-bda9-916ac5a127df tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:06.545214 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b868a9dc-c774-4ad5-bda9-916ac5a127df tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:18:06.545522 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b868a9dc-c774-4ad5-bda9-916ac5a127df tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:06.547149 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b868a9dc-c774-4ad5-bda9-916ac5a127df tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:06.558510 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1050/2082] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:18:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:18:06.562474 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa19218-a2ed-43cd-ad52-4f63f63e364d service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:06.572341 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa19218-a2ed-43cd-ad52-4f63f63e364d service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:06.577503 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ae1ac-fd73-41c1-b5a8-12dbe1de70c2 tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8653b4121a43429c85af72f699b35c74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b26c390f1a8c448aacd623d400d365ad', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:06.578160 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ae1ac-fd73-41c1-b5a8-12dbe1de70c2 tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:06.578449 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ae1ac-fd73-41c1-b5a8-12dbe1de70c2 tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:06.578732 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ae1ac-fd73-41c1-b5a8-12dbe1de70c2 tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:06.579484 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ae1ac-fd73-41c1-b5a8-12dbe1de70c2 tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:06.581387 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ae1ac-fd73-41c1-b5a8-12dbe1de70c2 tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:06.610313 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1033/2083] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:18:06 2022] POST /identity/v3/projects => generated 353 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:18:06.614559 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868a9dc-c774-4ad5-bda9-916ac5a127df tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:06.624522 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868a9dc-c774-4ad5-bda9-916ac5a127df tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:06.681257 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63282dfd-d1d2-41f7-aa08-01888b9ec85f tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8653b4121a43429c85af72f699b35c74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b26c390f1a8c448aacd623d400d365ad', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:06.682004 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63282dfd-d1d2-41f7-aa08-01888b9ec85f tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:06.682450 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63282dfd-d1d2-41f7-aa08-01888b9ec85f tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:06.682795 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63282dfd-d1d2-41f7-aa08-01888b9ec85f tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:06.683706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63282dfd-d1d2-41f7-aa08-01888b9ec85f tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:18:06.684095 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63282dfd-d1d2-41f7-aa08-01888b9ec85f tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:06.686011 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63282dfd-d1d2-41f7-aa08-01888b9ec85f tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:06.699150 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1051/2084] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:18:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:18:06.735955 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ae1ac-fd73-41c1-b5a8-12dbe1de70c2 tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:06.746247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ae1ac-fd73-41c1-b5a8-12dbe1de70c2 tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:06.750972 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d40bd6-2b9d-4907-afb8-3dc5a5d487a4 tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8653b4121a43429c85af72f699b35c74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b26c390f1a8c448aacd623d400d365ad', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:06.751890 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d40bd6-2b9d-4907-afb8-3dc5a5d487a4 tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:06.752181 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d40bd6-2b9d-4907-afb8-3dc5a5d487a4 tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:06.752461 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d40bd6-2b9d-4907-afb8-3dc5a5d487a4 tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] PATH_INFO: `/v3/projects/31f462e083a645d08c2c7127d63138db` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:06.753638 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d40bd6-2b9d-4907-afb8-3dc5a5d487a4 tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] RBAC: Authorizing `identity:delete_project(project_id=31f462e083a645d08c2c7127d63138db)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:06.755570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d40bd6-2b9d-4907-afb8-3dc5a5d487a4 tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:06.801594 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a7d40bd6-2b9d-4907-afb8-3dc5a5d487a4 tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31f462e083a645d08c2c7127d63138db', 'request_id': 'req-a7d40bd6-2b9d-4907-afb8-3dc5a5d487a4'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:06.803847 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=90,keystone:INSERT=6,keystone:DELETE=4 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:06.808875 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a7d40bd6-2b9d-4907-afb8-3dc5a5d487a4 tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31f462e083a645d08c2c7127d63138db', 'request_id': 'req-a7d40bd6-2b9d-4907-afb8-3dc5a5d487a4'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:06.818442 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a7d40bd6-2b9d-4907-afb8-3dc5a5d487a4 tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31f462e083a645d08c2c7127d63138db', 'request_id': 'req-a7d40bd6-2b9d-4907-afb8-3dc5a5d487a4'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:06.829649 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a7d40bd6-2b9d-4907-afb8-3dc5a5d487a4 tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] The token cache is being invalidate because project 31f462e083a645d08c2c7127d63138db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:18:06.829946 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a7d40bd6-2b9d-4907-afb8-3dc5a5d487a4 tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:06.830594 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a7d40bd6-2b9d-4907-afb8-3dc5a5d487a4 tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:06.832450 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1034/2085] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:18:06 2022] DELETE /identity/v3/projects/31f462e083a645d08c2c7127d63138db => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:18:07.026983 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63282dfd-d1d2-41f7-aa08-01888b9ec85f tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:07.028788 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-63282dfd-d1d2-41f7-aa08-01888b9ec85f tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:07.056219 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:07.101672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63282dfd-d1d2-41f7-aa08-01888b9ec85f tempest-VolumeQuotaClassesTest-1789438443 tempest-VolumeQuotaClassesTest-1789438443-project-admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:07.155700 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c1527-e7ea-478a-8622-d830e436e8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:07.156570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c1527-e7ea-478a-8622-d830e436e8f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:07.156853 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c1527-e7ea-478a-8622-d830e436e8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:07.157132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c1527-e7ea-478a-8622-d830e436e8f9 admin admin] PATH_INFO: `/v3/users/5f40166346f948b28986447ab392021c` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:07.158310 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c1527-e7ea-478a-8622-d830e436e8f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f40166346f948b28986447ab392021c)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:07.160374 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c1527-e7ea-478a-8622-d830e436e8f9 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:07.209364 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-e97c1527-e7ea-478a-8622-d830e436e8f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f40166346f948b28986447ab392021c'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:07.216773 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-e97c1527-e7ea-478a-8622-d830e436e8f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f40166346f948b28986447ab392021c'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:07.228865 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-e97c1527-e7ea-478a-8622-d830e436e8f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f40166346f948b28986447ab392021c'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:07.236573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-e97c1527-e7ea-478a-8622-d830e436e8f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f40166346f948b28986447ab392021c'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:07.237260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-e97c1527-e7ea-478a-8622-d830e436e8f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f40166346f948b28986447ab392021c'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:07.243038 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1052/2086] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:18:07 2022] DELETE /identity/v3/users/5f40166346f948b28986447ab392021c => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:18:07.248263 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5df7c-5e99-4f20-b707-54452ca0404a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:07.248707 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5df7c-5e99-4f20-b707-54452ca0404a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:07.248800 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5df7c-5e99-4f20-b707-54452ca0404a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:07.269431 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-83e5df7c-5e99-4f20-b707-54452ca0404a None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:07.270659 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-83e5df7c-5e99-4f20-b707-54452ca0404a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:07.328367 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1035/2087] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:07.337922 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c1527-e7ea-478a-8622-d830e436e8f9 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:07.339420 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-e97c1527-e7ea-478a-8622-d830e436e8f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:07.397852 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:07.404444 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c1527-e7ea-478a-8622-d830e436e8f9 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:07.409198 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f2129-bff8-452c-ad24-5a7f2737eedb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:07.409922 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f2129-bff8-452c-ad24-5a7f2737eedb service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:07.410207 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f2129-bff8-452c-ad24-5a7f2737eedb service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:07.410487 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f2129-bff8-452c-ad24-5a7f2737eedb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:07.420275 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188f2129-bff8-452c-ad24-5a7f2737eedb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:07.421710 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188f2129-bff8-452c-ad24-5a7f2737eedb service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:07.431242 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1053/2088] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:18:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:18:07.751422 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e5df7c-5e99-4f20-b707-54452ca0404a None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:07.752868 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-83e5df7c-5e99-4f20-b707-54452ca0404a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:07.775338 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e5df7c-5e99-4f20-b707-54452ca0404a None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:07.782857 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c61e9-a888-4ddc-a80f-07869ce020b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:07.783810 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c61e9-a888-4ddc-a80f-07869ce020b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:07.784157 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c61e9-a888-4ddc-a80f-07869ce020b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:07.784502 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c61e9-a888-4ddc-a80f-07869ce020b9 admin admin] PATH_INFO: `/v3/projects/d90dd72a1f1547239fc123a0f149cace` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:07.785827 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459c61e9-a888-4ddc-a80f-07869ce020b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d90dd72a1f1547239fc123a0f149cace)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:07.787861 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459c61e9-a888-4ddc-a80f-07869ce020b9 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:07.857210 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-459c61e9-a888-4ddc-a80f-07869ce020b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd90dd72a1f1547239fc123a0f149cace', 'request_id': 'req-459c61e9-a888-4ddc-a80f-07869ce020b9'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:07.864324 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-459c61e9-a888-4ddc-a80f-07869ce020b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd90dd72a1f1547239fc123a0f149cace', 'request_id': 'req-459c61e9-a888-4ddc-a80f-07869ce020b9'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:07.872627 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-459c61e9-a888-4ddc-a80f-07869ce020b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd90dd72a1f1547239fc123a0f149cace', 'request_id': 'req-459c61e9-a888-4ddc-a80f-07869ce020b9'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:07.885027 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-459c61e9-a888-4ddc-a80f-07869ce020b9 admin admin] The token cache is being invalidate because project d90dd72a1f1547239fc123a0f149cace was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:18:07.885325 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-459c61e9-a888-4ddc-a80f-07869ce020b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:07.886025 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-459c61e9-a888-4ddc-a80f-07869ce020b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:07.887686 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1036/2089] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:18:07 2022] DELETE /identity/v3/projects/d90dd72a1f1547239fc123a0f149cace => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:18:07.892471 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f2129-bff8-452c-ad24-5a7f2737eedb service neutron] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:07.905430 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f2129-bff8-452c-ad24-5a7f2737eedb service neutron] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:07.967342 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508f565-eb9b-4419-a4c9-740c06bcf76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:07.968220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508f565-eb9b-4419-a4c9-740c06bcf76a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:07.968501 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508f565-eb9b-4419-a4c9-740c06bcf76a admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:07.968774 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508f565-eb9b-4419-a4c9-740c06bcf76a admin admin] PATH_INFO: `/v3/users/8653b4121a43429c85af72f699b35c74` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:07.969933 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e508f565-eb9b-4419-a4c9-740c06bcf76a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8653b4121a43429c85af72f699b35c74)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:07.971864 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e508f565-eb9b-4419-a4c9-740c06bcf76a admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:08.015382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-e508f565-eb9b-4419-a4c9-740c06bcf76a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8653b4121a43429c85af72f699b35c74'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:08.022968 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-e508f565-eb9b-4419-a4c9-740c06bcf76a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8653b4121a43429c85af72f699b35c74'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:08.035855 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-e508f565-eb9b-4419-a4c9-740c06bcf76a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8653b4121a43429c85af72f699b35c74'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:08.042564 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-e508f565-eb9b-4419-a4c9-740c06bcf76a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8653b4121a43429c85af72f699b35c74'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:08.043223 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-e508f565-eb9b-4419-a4c9-740c06bcf76a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8653b4121a43429c85af72f699b35c74'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:08.051331 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1054/2090] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:18:07 2022] DELETE /identity/v3/users/8653b4121a43429c85af72f699b35c74 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:18:08.327248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c61e9-a888-4ddc-a80f-07869ce020b9 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:08.328785 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-459c61e9-a888-4ddc-a80f-07869ce020b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:08.344667 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c61e9-a888-4ddc-a80f-07869ce020b9 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:08.408007 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb92ff-ee37-4079-832e-0c072ea51c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:08.408975 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb92ff-ee37-4079-832e-0c072ea51c50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:08.409264 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb92ff-ee37-4079-832e-0c072ea51c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:08.409542 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb92ff-ee37-4079-832e-0c072ea51c50 admin admin] PATH_INFO: `/v3/projects/b26c390f1a8c448aacd623d400d365ad` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:08.410711 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb92ff-ee37-4079-832e-0c072ea51c50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b26c390f1a8c448aacd623d400d365ad)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:08.412641 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb92ff-ee37-4079-832e-0c072ea51c50 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:08.418644 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:08.459021 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5cfb92ff-ee37-4079-832e-0c072ea51c50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b26c390f1a8c448aacd623d400d365ad', 'request_id': 'req-5cfb92ff-ee37-4079-832e-0c072ea51c50'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:08.465239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5cfb92ff-ee37-4079-832e-0c072ea51c50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b26c390f1a8c448aacd623d400d365ad', 'request_id': 'req-5cfb92ff-ee37-4079-832e-0c072ea51c50'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:08.473576 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5cfb92ff-ee37-4079-832e-0c072ea51c50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b26c390f1a8c448aacd623d400d365ad', 'request_id': 'req-5cfb92ff-ee37-4079-832e-0c072ea51c50'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:08.485971 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5cfb92ff-ee37-4079-832e-0c072ea51c50 admin admin] The token cache is being invalidate because project b26c390f1a8c448aacd623d400d365ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:18:08.486263 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5cfb92ff-ee37-4079-832e-0c072ea51c50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:08.486921 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5cfb92ff-ee37-4079-832e-0c072ea51c50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:08.488525 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1037/2091] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:18:08 2022] DELETE /identity/v3/projects/b26c390f1a8c448aacd623d400d365ad => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:18:08.493370 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070f454-6036-48f7-be8a-13e4c9f41ef9 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:08.493652 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070f454-6036-48f7-be8a-13e4c9f41ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:08.493928 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070f454-6036-48f7-be8a-13e4c9f41ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:08.513057 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-4070f454-6036-48f7-be8a-13e4c9f41ef9 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:08.514290 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-4070f454-6036-48f7-be8a-13e4c9f41ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:08.575201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1055/2092] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:08.582732 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8b037-ef1e-43c0-876e-883299b99520 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:08.583030 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8b037-ef1e-43c0-876e-883299b99520 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:08.583335 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8b037-ef1e-43c0-876e-883299b99520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:08.601597 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-cdc8b037-ef1e-43c0-876e-883299b99520 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:08.602994 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-cdc8b037-ef1e-43c0-876e-883299b99520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:08.611088 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1038/2093] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:08.615343 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070f454-6036-48f7-be8a-13e4c9f41ef9 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:08.625519 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070f454-6036-48f7-be8a-13e4c9f41ef9 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:08.630368 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3324bf9a-2a8e-4a36-a799-68e96d176647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:08.631048 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324bf9a-2a8e-4a36-a799-68e96d176647 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:08.631361 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324bf9a-2a8e-4a36-a799-68e96d176647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:08.631642 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324bf9a-2a8e-4a36-a799-68e96d176647 admin admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:08.632533 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3324bf9a-2a8e-4a36-a799-68e96d176647 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:18:08.632846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3324bf9a-2a8e-4a36-a799-68e96d176647 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:08.634465 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3324bf9a-2a8e-4a36-a799-68e96d176647 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:08.639708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:08.656198 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1056/2094] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:18:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:18:08.656831 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc8b037-ef1e-43c0-876e-883299b99520 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:08.666281 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc8b037-ef1e-43c0-876e-883299b99520 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:08.671856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4823acf-b2ad-4242-b834-8a7dbe867046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:08.672582 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4823acf-b2ad-4242-b834-8a7dbe867046 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:08.672940 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4823acf-b2ad-4242-b834-8a7dbe867046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:08.673285 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4823acf-b2ad-4242-b834-8a7dbe867046 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:08.674163 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4823acf-b2ad-4242-b834-8a7dbe867046 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:08.676194 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4823acf-b2ad-4242-b834-8a7dbe867046 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:08.704670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1039/2095] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:18:08 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:18:08.708958 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3324bf9a-2a8e-4a36-a799-68e96d176647 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:08.719557 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3324bf9a-2a8e-4a36-a799-68e96d176647 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:08.778943 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f236524-3961-4c35-b5d9-4feae667a491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:08.779645 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f236524-3961-4c35-b5d9-4feae667a491 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:08.779933 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f236524-3961-4c35-b5d9-4feae667a491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:08.780214 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f236524-3961-4c35-b5d9-4feae667a491 admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:08.780932 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f236524-3961-4c35-b5d9-4feae667a491 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:08.782860 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f236524-3961-4c35-b5d9-4feae667a491 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:08.802734 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1057/2096] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:18:08 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:18:08.806853 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4823acf-b2ad-4242-b834-8a7dbe867046 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:08.817694 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4823acf-b2ad-4242-b834-8a7dbe867046 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:08.822397 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340c24a-4041-4b8e-8b6b-ea501f1cbbf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:08.823054 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340c24a-4041-4b8e-8b6b-ea501f1cbbf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:08.823374 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340c24a-4041-4b8e-8b6b-ea501f1cbbf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:08.823657 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340c24a-4041-4b8e-8b6b-ea501f1cbbf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:08.824395 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2340c24a-4041-4b8e-8b6b-ea501f1cbbf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:08.825995 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2340c24a-4041-4b8e-8b6b-ea501f1cbbf1 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:08.836540 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1040/2097] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:18:08.840261 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f236524-3961-4c35-b5d9-4feae667a491 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:08.849989 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f236524-3961-4c35-b5d9-4feae667a491 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:08.855146 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc634a6d-a629-479f-b333-7e278b27fcd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:08.855839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc634a6d-a629-479f-b333-7e278b27fcd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:08.856124 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc634a6d-a629-479f-b333-7e278b27fcd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:08.856403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc634a6d-a629-479f-b333-7e278b27fcd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:08.857131 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc634a6d-a629-479f-b333-7e278b27fcd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:08.858729 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc634a6d-a629-479f-b333-7e278b27fcd5 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:08.872789 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1058/2098] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:18:08.876505 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340c24a-4041-4b8e-8b6b-ea501f1cbbf1 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:08.886484 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340c24a-4041-4b8e-8b6b-ea501f1cbbf1 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:08.891028 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464371f8-ffc7-4c62-8c83-5334cf1b7c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:08.891932 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464371f8-ffc7-4c62-8c83-5334cf1b7c3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:08.892216 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464371f8-ffc7-4c62-8c83-5334cf1b7c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:08.892492 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464371f8-ffc7-4c62-8c83-5334cf1b7c3e admin admin] PATH_INFO: `/v3/projects/8ad6d2af9ad84ecd884e3a5658464cab/users/fe96eb22bbe145c48ac08d07c8889eb2/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:08.906603 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464371f8-ffc7-4c62-8c83-5334cf1b7c3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ad6d2af9ad84ecd884e3a5658464cab, user_id=fe96eb22bbe145c48ac08d07c8889eb2, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:08.909262 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464371f8-ffc7-4c62-8c83-5334cf1b7c3e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:08.921989 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1041/2099] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:18:08 2022] PUT /identity/v3/projects/8ad6d2af9ad84ecd884e3a5658464cab/users/fe96eb22bbe145c48ac08d07c8889eb2/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:18:08.931940 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd5deb-b94b-41a9-ad4f-ff356aa8afc6 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:08.932273 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd5deb-b94b-41a9-ad4f-ff356aa8afc6 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:08.932635 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd5deb-b94b-41a9-ad4f-ff356aa8afc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:08.959315 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-6cdd5deb-b94b-41a9-ad4f-ff356aa8afc6 None None] MFA Rules not processed for user `fe96eb22bbe145c48ac08d07c8889eb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:08.960521 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-6cdd5deb-b94b-41a9-ad4f-ff356aa8afc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:09.030850 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1059/2100] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:09.041348 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464371f8-ffc7-4c62-8c83-5334cf1b7c3e admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:09.043614 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-464371f8-ffc7-4c62-8c83-5334cf1b7c3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:09.108156 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464371f8-ffc7-4c62-8c83-5334cf1b7c3e admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:09.112108 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b0ed5-f8c2-46fc-8ab6-74d954c01997 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:09.112730 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b0ed5-f8c2-46fc-8ab6-74d954c01997 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:09.113015 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b0ed5-f8c2-46fc-8ab6-74d954c01997 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:09.113413 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b0ed5-f8c2-46fc-8ab6-74d954c01997 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:09.122571 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8b0ed5-f8c2-46fc-8ab6-74d954c01997 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:09.124050 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8b0ed5-f8c2-46fc-8ab6-74d954c01997 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:09.134401 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1042/2101] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:18:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:18:09.166199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd5deb-b94b-41a9-ad4f-ff356aa8afc6 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:09.229147 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd5deb-b94b-41a9-ad4f-ff356aa8afc6 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:09.234656 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f5bc4a-7469-4a41-a0d5-f3dbc9c0e62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:09.235573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f5bc4a-7469-4a41-a0d5-f3dbc9c0e62e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:09.235863 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f5bc4a-7469-4a41-a0d5-f3dbc9c0e62e admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:09.236139 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f5bc4a-7469-4a41-a0d5-f3dbc9c0e62e admin admin] PATH_INFO: `/v3/users/fe96eb22bbe145c48ac08d07c8889eb2` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:09.237365 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f5bc4a-7469-4a41-a0d5-f3dbc9c0e62e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe96eb22bbe145c48ac08d07c8889eb2)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:09.239408 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f5bc4a-7469-4a41-a0d5-f3dbc9c0e62e admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:09.244280 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:09.304413 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-22f5bc4a-7469-4a41-a0d5-f3dbc9c0e62e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe96eb22bbe145c48ac08d07c8889eb2'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:09.313023 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-22f5bc4a-7469-4a41-a0d5-f3dbc9c0e62e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe96eb22bbe145c48ac08d07c8889eb2'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:09.328289 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-22f5bc4a-7469-4a41-a0d5-f3dbc9c0e62e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe96eb22bbe145c48ac08d07c8889eb2'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:09.335611 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-22f5bc4a-7469-4a41-a0d5-f3dbc9c0e62e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe96eb22bbe145c48ac08d07c8889eb2'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:09.336327 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-22f5bc4a-7469-4a41-a0d5-f3dbc9c0e62e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe96eb22bbe145c48ac08d07c8889eb2'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:09.343130 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1060/2102] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:18:09 2022] DELETE /identity/v3/users/fe96eb22bbe145c48ac08d07c8889eb2 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:18:09.347561 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4248a8-2bfc-4b58-9446-b4df3f3dde99 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:09.348041 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4248a8-2bfc-4b58-9446-b4df3f3dde99 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:09.348148 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4248a8-2bfc-4b58-9446-b4df3f3dde99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:09.366632 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-ef4248a8-2bfc-4b58-9446-b4df3f3dde99 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:09.368023 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ef4248a8-2bfc-4b58-9446-b4df3f3dde99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:09.400160 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:09.432920 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1043/2103] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:18:09.442247 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f5bc4a-7469-4a41-a0d5-f3dbc9c0e62e admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:09.443693 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-22f5bc4a-7469-4a41-a0d5-f3dbc9c0e62e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:09.499493 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f5bc4a-7469-4a41-a0d5-f3dbc9c0e62e admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:09.503041 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca920f6-a830-4faf-9d29-ade4a715bc5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:09.503712 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca920f6-a830-4faf-9d29-ade4a715bc5d service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:09.503995 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca920f6-a830-4faf-9d29-ade4a715bc5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:09.504269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca920f6-a830-4faf-9d29-ade4a715bc5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:09.513478 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca920f6-a830-4faf-9d29-ade4a715bc5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:09.514894 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca920f6-a830-4faf-9d29-ade4a715bc5d service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:09.524056 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1061/2104] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:18:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:18:09.785088 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4248a8-2bfc-4b58-9446-b4df3f3dde99 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:09.786621 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ef4248a8-2bfc-4b58-9446-b4df3f3dde99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:09.813393 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4248a8-2bfc-4b58-9446-b4df3f3dde99 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:09.818139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0301ec44-4825-4304-9aa6-ae170ba9b907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:09.819131 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301ec44-4825-4304-9aa6-ae170ba9b907 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:09.819508 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301ec44-4825-4304-9aa6-ae170ba9b907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:09.819855 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301ec44-4825-4304-9aa6-ae170ba9b907 admin admin] PATH_INFO: `/v3/projects/8ad6d2af9ad84ecd884e3a5658464cab` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:09.821114 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0301ec44-4825-4304-9aa6-ae170ba9b907 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ad6d2af9ad84ecd884e3a5658464cab)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:09.823187 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0301ec44-4825-4304-9aa6-ae170ba9b907 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:09.873625 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0301ec44-4825-4304-9aa6-ae170ba9b907 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ad6d2af9ad84ecd884e3a5658464cab', 'request_id': 'req-0301ec44-4825-4304-9aa6-ae170ba9b907'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:09.880067 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0301ec44-4825-4304-9aa6-ae170ba9b907 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ad6d2af9ad84ecd884e3a5658464cab', 'request_id': 'req-0301ec44-4825-4304-9aa6-ae170ba9b907'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:09.888930 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0301ec44-4825-4304-9aa6-ae170ba9b907 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ad6d2af9ad84ecd884e3a5658464cab', 'request_id': 'req-0301ec44-4825-4304-9aa6-ae170ba9b907'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:09.908111 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0301ec44-4825-4304-9aa6-ae170ba9b907 admin admin] The token cache is being invalidate because project 8ad6d2af9ad84ecd884e3a5658464cab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:18:09.908407 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0301ec44-4825-4304-9aa6-ae170ba9b907 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:09.909078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0301ec44-4825-4304-9aa6-ae170ba9b907 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:09.910738 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1044/2105] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:18:09 2022] DELETE /identity/v3/projects/8ad6d2af9ad84ecd884e3a5658464cab => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:18:09.915536 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0fdd0f-29f2-4bb8-a08f-24e81d418528 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:09.915835 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0fdd0f-29f2-4bb8-a08f-24e81d418528 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:09.916118 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0fdd0f-29f2-4bb8-a08f-24e81d418528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:09.944004 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-8b0fdd0f-29f2-4bb8-a08f-24e81d418528 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:09.945196 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-8b0fdd0f-29f2-4bb8-a08f-24e81d418528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:10.002435 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1062/2106] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:18:10.011498 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b0f2b-da86-4e7b-9505-4701cc371ba1 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:10.011806 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b0f2b-da86-4e7b-9505-4701cc371ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:10.012085 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b0f2b-da86-4e7b-9505-4701cc371ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:10.032471 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-d66b0f2b-da86-4e7b-9505-4701cc371ba1 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:10.033715 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d66b0f2b-da86-4e7b-9505-4701cc371ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:10.041755 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1045/2107] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:10.046096 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0fdd0f-29f2-4bb8-a08f-24e81d418528 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:10.055702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0fdd0f-29f2-4bb8-a08f-24e81d418528 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:10.060795 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f923a7be-922d-4025-b49b-cd27ce43ac97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:10.061446 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923a7be-922d-4025-b49b-cd27ce43ac97 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:10.061728 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923a7be-922d-4025-b49b-cd27ce43ac97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:10.061998 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923a7be-922d-4025-b49b-cd27ce43ac97 admin admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:10.062717 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f923a7be-922d-4025-b49b-cd27ce43ac97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:18:10.063017 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f923a7be-922d-4025-b49b-cd27ce43ac97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:10.064624 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f923a7be-922d-4025-b49b-cd27ce43ac97 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:10.076793 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1063/2108] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:18:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:18:10.081265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66b0f2b-da86-4e7b-9505-4701cc371ba1 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:10.090393 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66b0f2b-da86-4e7b-9505-4701cc371ba1 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:10.095070 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065f9964-cbf1-42e6-9338-2c7b54485849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:10.095743 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f9964-cbf1-42e6-9338-2c7b54485849 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:10.096027 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f9964-cbf1-42e6-9338-2c7b54485849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:10.096303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f9964-cbf1-42e6-9338-2c7b54485849 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:10.096994 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065f9964-cbf1-42e6-9338-2c7b54485849 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:10.098916 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065f9964-cbf1-42e6-9338-2c7b54485849 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:10.123861 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1046/2109] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:18:10 2022] POST /identity/v3/projects => generated 377 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:18:10.127777 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f923a7be-922d-4025-b49b-cd27ce43ac97 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:10.134297 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:10.138824 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f923a7be-922d-4025-b49b-cd27ce43ac97 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:10.202260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a3abf1-8bce-4ff4-8486-01d5bfc02f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:10.202932 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3abf1-8bce-4ff4-8486-01d5bfc02f27 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:10.203219 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3abf1-8bce-4ff4-8486-01d5bfc02f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:10.203534 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3abf1-8bce-4ff4-8486-01d5bfc02f27 admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:10.204229 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a3abf1-8bce-4ff4-8486-01d5bfc02f27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:10.206127 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a3abf1-8bce-4ff4-8486-01d5bfc02f27 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:10.229308 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1064/2110] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:18:10 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:18:10.231008 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065f9964-cbf1-42e6-9338-2c7b54485849 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:10.241260 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065f9964-cbf1-42e6-9338-2c7b54485849 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:10.246478 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94730e71-1c33-40d2-a7d2-c762fdef54cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:10.247136 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94730e71-1c33-40d2-a7d2-c762fdef54cc admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:10.247457 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94730e71-1c33-40d2-a7d2-c762fdef54cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:10.247738 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94730e71-1c33-40d2-a7d2-c762fdef54cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:10.248457 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94730e71-1c33-40d2-a7d2-c762fdef54cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:10.250045 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94730e71-1c33-40d2-a7d2-c762fdef54cc admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:10.261304 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1047/2111] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:18:10.265333 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a3abf1-8bce-4ff4-8486-01d5bfc02f27 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:10.275133 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a3abf1-8bce-4ff4-8486-01d5bfc02f27 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:10.280447 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd9947-7e41-4068-85fd-84d348be82c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:10.281156 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd9947-7e41-4068-85fd-84d348be82c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:10.281471 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd9947-7e41-4068-85fd-84d348be82c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:10.281754 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd9947-7e41-4068-85fd-84d348be82c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:10.282476 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dd9947-7e41-4068-85fd-84d348be82c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:10.284107 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dd9947-7e41-4068-85fd-84d348be82c5 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:10.294620 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1065/2112] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:18:10.298550 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94730e71-1c33-40d2-a7d2-c762fdef54cc admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:10.309214 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94730e71-1c33-40d2-a7d2-c762fdef54cc admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:10.313842 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d022df-4e27-429d-ae9c-bc7d0336c9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:10.314727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d022df-4e27-429d-ae9c-bc7d0336c9c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:10.315009 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d022df-4e27-429d-ae9c-bc7d0336c9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:10.315282 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d022df-4e27-429d-ae9c-bc7d0336c9c5 admin admin] PATH_INFO: `/v3/projects/91ee9661fe3c4c64849af4d4acce90db/users/9d9b356f076b45aebccd20245c652437/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:10.329943 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d022df-4e27-429d-ae9c-bc7d0336c9c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91ee9661fe3c4c64849af4d4acce90db, user_id=9d9b356f076b45aebccd20245c652437, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:10.332615 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d022df-4e27-429d-ae9c-bc7d0336c9c5 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:10.345549 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1048/2113] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:18:10 2022] PUT /identity/v3/projects/91ee9661fe3c4c64849af4d4acce90db/users/9d9b356f076b45aebccd20245c652437/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:18:10.356160 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d376fe-c6e4-46ac-b7ef-5a72e7fe3cd5 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:10.356454 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d376fe-c6e4-46ac-b7ef-5a72e7fe3cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:10.356737 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d376fe-c6e4-46ac-b7ef-5a72e7fe3cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:10.387329 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-e6d376fe-c6e4-46ac-b7ef-5a72e7fe3cd5 None None] MFA Rules not processed for user `9d9b356f076b45aebccd20245c652437`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:10.389058 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-e6d376fe-c6e4-46ac-b7ef-5a72e7fe3cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:10.464320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1066/2114] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:10.474907 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d022df-4e27-429d-ae9c-bc7d0336c9c5 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:10.476600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-96d022df-4e27-429d-ae9c-bc7d0336c9c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:10.534760 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d022df-4e27-429d-ae9c-bc7d0336c9c5 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:10.538312 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48c124-20b4-499e-b3a5-ee3d393d2642 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbaf66d6e1f54047917cc615546bbd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:10.538934 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48c124-20b4-499e-b3a5-ee3d393d2642 service glance] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:10.539216 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48c124-20b4-499e-b3a5-ee3d393d2642 service glance] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:10.539527 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48c124-20b4-499e-b3a5-ee3d393d2642 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:10.546415 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:10.548881 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48c124-20b4-499e-b3a5-ee3d393d2642 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:10.550976 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48c124-20b4-499e-b3a5-ee3d393d2642 service glance] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:10.562548 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1049/2115] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:18:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:18:17.562963 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3504aa-59f0-4a2e-a8b6-c9b5d237fd70 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:17.563596 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3504aa-59f0-4a2e-a8b6-c9b5d237fd70 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:17.563596 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3504aa-59f0-4a2e-a8b6-c9b5d237fd70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:17.598983 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-0b3504aa-59f0-4a2e-a8b6-c9b5d237fd70 None None] MFA Rules not processed for user `875140cf51d9410da45f8c3805b26204`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:17.600230 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-0b3504aa-59f0-4a2e-a8b6-c9b5d237fd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:17.655823 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1067/2116] 10.222.0.41 () {58 vars in 1009 bytes} [Fri Jun 3 16:18:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:17.695748 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48c124-20b4-499e-b3a5-ee3d393d2642 service glance] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:17.697154 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3f48c124-20b4-499e-b3a5-ee3d393d2642 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:17.765661 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48c124-20b4-499e-b3a5-ee3d393d2642 service glance] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:17.770650 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7716770c-adec-4e7d-9c90-d2776fc528bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5f513b781a4a73b443cded13359eda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:17.771288 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716770c-adec-4e7d-9c90-d2776fc528bc service swift] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:17.771604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716770c-adec-4e7d-9c90-d2776fc528bc service swift] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:17.771882 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716770c-adec-4e7d-9c90-d2776fc528bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:17.779999 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7716770c-adec-4e7d-9c90-d2776fc528bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:17.781572 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7716770c-adec-4e7d-9c90-d2776fc528bc service swift] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:17.789429 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1050/2117] 10.222.0.41 () {60 vars in 1384 bytes} [Fri Jun 3 16:18:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 16:18:18.300501 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a73ef-7195-4f59-9249-55214d009063 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:18.300799 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a73ef-7195-4f59-9249-55214d009063 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:18.301091 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a73ef-7195-4f59-9249-55214d009063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:18.318910 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-627a73ef-7195-4f59-9249-55214d009063 None None] MFA Rules not processed for user `875140cf51d9410da45f8c3805b26204`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:18.320139 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-627a73ef-7195-4f59-9249-55214d009063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:18.328281 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1068/2118] 10.222.0.41 () {58 vars in 967 bytes} [Fri Jun 3 16:18:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:18.340322 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7716770c-adec-4e7d-9c90-d2776fc528bc service swift] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:18.351126 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7716770c-adec-4e7d-9c90-d2776fc528bc service swift] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:18.356410 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e46ff1f-1eb5-411c-bf37-8e049e3333ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5f513b781a4a73b443cded13359eda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:18.357039 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46ff1f-1eb5-411c-bf37-8e049e3333ef service swift] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:18.357320 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46ff1f-1eb5-411c-bf37-8e049e3333ef service swift] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:18.357603 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46ff1f-1eb5-411c-bf37-8e049e3333ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:18.365598 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e46ff1f-1eb5-411c-bf37-8e049e3333ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:18.367162 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e46ff1f-1eb5-411c-bf37-8e049e3333ef service swift] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:18.375821 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1051/2119] 10.222.0.41 () {60 vars in 1384 bytes} [Fri Jun 3 16:18:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 16:18:18.581043 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c2b45-136f-4bd6-8e99-6b6f4331bc88 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:18.581516 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c2b45-136f-4bd6-8e99-6b6f4331bc88 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:18.581741 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c2b45-136f-4bd6-8e99-6b6f4331bc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:18.594350 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:18.604881 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-106c2b45-136f-4bd6-8e99-6b6f4331bc88 None None] MFA Rules not processed for user `875140cf51d9410da45f8c3805b26204`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:18.606139 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-106c2b45-136f-4bd6-8e99-6b6f4331bc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:18.613636 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1069/2120] 10.222.0.41 () {58 vars in 1009 bytes} [Fri Jun 3 16:18:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:18.625651 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e46ff1f-1eb5-411c-bf37-8e049e3333ef service swift] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:18.637209 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e46ff1f-1eb5-411c-bf37-8e049e3333ef service swift] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:18.642485 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b035364-65f8-4081-9789-aa7e12fc4993 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5f513b781a4a73b443cded13359eda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:18.643137 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b035364-65f8-4081-9789-aa7e12fc4993 service swift] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:18.643464 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b035364-65f8-4081-9789-aa7e12fc4993 service swift] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:18.643743 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b035364-65f8-4081-9789-aa7e12fc4993 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:18.653345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b035364-65f8-4081-9789-aa7e12fc4993 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:18.654921 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b035364-65f8-4081-9789-aa7e12fc4993 service swift] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:18.662801 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1052/2121] 10.222.0.41 () {60 vars in 1384 bytes} [Fri Jun 3 16:18:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 16:18:18.968147 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01130576-78b4-4493-818e-7eae286686bc None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:18.968435 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01130576-78b4-4493-818e-7eae286686bc None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:18.968712 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01130576-78b4-4493-818e-7eae286686bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:18.985693 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-01130576-78b4-4493-818e-7eae286686bc None None] MFA Rules not processed for user `875140cf51d9410da45f8c3805b26204`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:18.986865 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-01130576-78b4-4493-818e-7eae286686bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:18.993886 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1070/2122] 10.222.0.41 () {58 vars in 967 bytes} [Fri Jun 3 16:18:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:19.007643 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b035364-65f8-4081-9789-aa7e12fc4993 service swift] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:19.018962 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b035364-65f8-4081-9789-aa7e12fc4993 service swift] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:19.024174 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bad1a0f-2516-4f0b-bf12-067ce62c5d4e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5f513b781a4a73b443cded13359eda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:19.024839 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad1a0f-2516-4f0b-bf12-067ce62c5d4e service swift] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:19.025128 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad1a0f-2516-4f0b-bf12-067ce62c5d4e service swift] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:19.025405 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad1a0f-2516-4f0b-bf12-067ce62c5d4e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:19.034022 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bad1a0f-2516-4f0b-bf12-067ce62c5d4e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:19.035644 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bad1a0f-2516-4f0b-bf12-067ce62c5d4e service swift] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:19.043583 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1053/2123] 10.222.0.41 () {60 vars in 1384 bytes} [Fri Jun 3 16:18:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 16:18:21.094904 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01130576-78b4-4493-818e-7eae286686bc None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:21.096335 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-01130576-78b4-4493-818e-7eae286686bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:21.162995 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01130576-78b4-4493-818e-7eae286686bc None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:21.167494 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ddb59-36af-4ef9-86c5-93cffbc7e78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:21.168366 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ddb59-36af-4ef9-86c5-93cffbc7e78b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:21.168650 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ddb59-36af-4ef9-86c5-93cffbc7e78b admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:21.168919 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ddb59-36af-4ef9-86c5-93cffbc7e78b admin admin] PATH_INFO: `/v3/users/334e442aa8d348beaa8dde1f78f5c68d` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:21.170105 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964ddb59-36af-4ef9-86c5-93cffbc7e78b admin admin] RBAC: Authorizing `identity:delete_user(user_id=334e442aa8d348beaa8dde1f78f5c68d)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:21.172120 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964ddb59-36af-4ef9-86c5-93cffbc7e78b admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:21.229969 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-964ddb59-36af-4ef9-86c5-93cffbc7e78b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '334e442aa8d348beaa8dde1f78f5c68d'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:21.237766 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-964ddb59-36af-4ef9-86c5-93cffbc7e78b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '334e442aa8d348beaa8dde1f78f5c68d'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:21.251441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-964ddb59-36af-4ef9-86c5-93cffbc7e78b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '334e442aa8d348beaa8dde1f78f5c68d'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:21.258747 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-964ddb59-36af-4ef9-86c5-93cffbc7e78b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '334e442aa8d348beaa8dde1f78f5c68d'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:21.259506 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-964ddb59-36af-4ef9-86c5-93cffbc7e78b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '334e442aa8d348beaa8dde1f78f5c68d'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:21.265981 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1071/2124] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:18:21 2022] DELETE /identity/v3/users/334e442aa8d348beaa8dde1f78f5c68d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:18:21.270524 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a23f8-b9af-4bb0-8b17-660ad62ec99c None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:21.271009 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a23f8-b9af-4bb0-8b17-660ad62ec99c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:21.271102 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a23f8-b9af-4bb0-8b17-660ad62ec99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:21.288878 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-754a23f8-b9af-4bb0-8b17-660ad62ec99c None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:21.290129 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-754a23f8-b9af-4bb0-8b17-660ad62ec99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:21.353843 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1054/2125] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:21.365803 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ddb59-36af-4ef9-86c5-93cffbc7e78b admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:21.367380 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-964ddb59-36af-4ef9-86c5-93cffbc7e78b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:21.426684 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ddb59-36af-4ef9-86c5-93cffbc7e78b admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:21.430502 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c2ec03-53ad-465c-9474-1b0267b48eff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:21.431229 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2ec03-53ad-465c-9474-1b0267b48eff service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:21.431606 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2ec03-53ad-465c-9474-1b0267b48eff service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:21.431888 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2ec03-53ad-465c-9474-1b0267b48eff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:21.442910 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c2ec03-53ad-465c-9474-1b0267b48eff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:21.444600 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c2ec03-53ad-465c-9474-1b0267b48eff service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:21.455280 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1072/2126] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:18:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:18:21.625346 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a23f8-b9af-4bb0-8b17-660ad62ec99c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:21.626792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-754a23f8-b9af-4bb0-8b17-660ad62ec99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:21.638489 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a23f8-b9af-4bb0-8b17-660ad62ec99c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:21.643184 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec150e4-798b-4599-a545-bc179a1b931c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:21.644096 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec150e4-798b-4599-a545-bc179a1b931c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:21.644379 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec150e4-798b-4599-a545-bc179a1b931c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:21.644659 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec150e4-798b-4599-a545-bc179a1b931c admin admin] PATH_INFO: `/v3/projects/09848a8d962b47439b233d1f06bf72e3` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:21.646007 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec150e4-798b-4599-a545-bc179a1b931c admin admin] RBAC: Authorizing `identity:delete_project(project_id=09848a8d962b47439b233d1f06bf72e3)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:21.655140 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec150e4-798b-4599-a545-bc179a1b931c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:21.698382 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:21.705593 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2ec150e4-798b-4599-a545-bc179a1b931c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09848a8d962b47439b233d1f06bf72e3', 'request_id': 'req-2ec150e4-798b-4599-a545-bc179a1b931c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:21.711845 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2ec150e4-798b-4599-a545-bc179a1b931c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09848a8d962b47439b233d1f06bf72e3', 'request_id': 'req-2ec150e4-798b-4599-a545-bc179a1b931c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:21.719473 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2ec150e4-798b-4599-a545-bc179a1b931c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09848a8d962b47439b233d1f06bf72e3', 'request_id': 'req-2ec150e4-798b-4599-a545-bc179a1b931c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:21.729513 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2ec150e4-798b-4599-a545-bc179a1b931c admin admin] The token cache is being invalidate because project 09848a8d962b47439b233d1f06bf72e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:18:21.729806 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2ec150e4-798b-4599-a545-bc179a1b931c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:21.730453 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2ec150e4-798b-4599-a545-bc179a1b931c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:21.732512 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1055/2127] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:18:21 2022] DELETE /identity/v3/projects/09848a8d962b47439b233d1f06bf72e3 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:18:21.737544 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd26159-591e-4eb4-9970-99bbfd477836 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:21.737839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd26159-591e-4eb4-9970-99bbfd477836 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:21.738121 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd26159-591e-4eb4-9970-99bbfd477836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:21.757948 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-1cd26159-591e-4eb4-9970-99bbfd477836 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:21.759135 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-1cd26159-591e-4eb4-9970-99bbfd477836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:21.764340 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:21.820641 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1073/2128] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:21.829190 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40952d-8497-40f3-9fd4-04daef64ce5f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:21.829698 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40952d-8497-40f3-9fd4-04daef64ce5f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:21.830076 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40952d-8497-40f3-9fd4-04daef64ce5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:21.850101 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-5b40952d-8497-40f3-9fd4-04daef64ce5f None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:21.851340 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5b40952d-8497-40f3-9fd4-04daef64ce5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:21.859977 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1056/2129] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:21.864204 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd26159-591e-4eb4-9970-99bbfd477836 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:21.874067 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd26159-591e-4eb4-9970-99bbfd477836 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:21.878722 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d533f80-bb68-44d7-bf10-19e88190db51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:21.879416 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d533f80-bb68-44d7-bf10-19e88190db51 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:21.879707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d533f80-bb68-44d7-bf10-19e88190db51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:21.879989 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d533f80-bb68-44d7-bf10-19e88190db51 admin admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:21.880719 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d533f80-bb68-44d7-bf10-19e88190db51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:18:21.881046 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d533f80-bb68-44d7-bf10-19e88190db51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:21.882655 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d533f80-bb68-44d7-bf10-19e88190db51 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:21.894108 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1074/2130] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:18:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:18:21.898236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b40952d-8497-40f3-9fd4-04daef64ce5f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:21.908152 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b40952d-8497-40f3-9fd4-04daef64ce5f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:21.913165 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c34aa9-b846-4855-b525-340ccd445154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:21.913807 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c34aa9-b846-4855-b525-340ccd445154 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:21.914095 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c34aa9-b846-4855-b525-340ccd445154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:21.914377 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c34aa9-b846-4855-b525-340ccd445154 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:21.915074 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c34aa9-b846-4855-b525-340ccd445154 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:21.917016 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c34aa9-b846-4855-b525-340ccd445154 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:21.941886 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1057/2131] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:18:21 2022] POST /identity/v3/projects => generated 379 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:18:21.946031 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d533f80-bb68-44d7-bf10-19e88190db51 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:21.956047 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d533f80-bb68-44d7-bf10-19e88190db51 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:22.011739 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c7fc2-043d-4259-bba7-ea625412c26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:22.012398 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c7fc2-043d-4259-bba7-ea625412c26a admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:22.012682 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c7fc2-043d-4259-bba7-ea625412c26a admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:22.012954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c7fc2-043d-4259-bba7-ea625412c26a admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:22.013629 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052c7fc2-043d-4259-bba7-ea625412c26a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:22.015576 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052c7fc2-043d-4259-bba7-ea625412c26a admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:22.039116 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1075/2132] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:18:21 2022] POST /identity/v3/users => generated 290 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:18:22.040655 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c34aa9-b846-4855-b525-340ccd445154 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:22.052575 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c34aa9-b846-4855-b525-340ccd445154 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:22.057379 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0944785-cfce-4005-818d-7811e00cd67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:22.058044 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0944785-cfce-4005-818d-7811e00cd67b admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:22.058325 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0944785-cfce-4005-818d-7811e00cd67b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:22.058601 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0944785-cfce-4005-818d-7811e00cd67b admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:22.059357 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0944785-cfce-4005-818d-7811e00cd67b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:22.060953 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0944785-cfce-4005-818d-7811e00cd67b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:22.072515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1058/2133] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:18:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:18:22.076752 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c7fc2-043d-4259-bba7-ea625412c26a admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:22.086950 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c7fc2-043d-4259-bba7-ea625412c26a admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:22.092221 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec546d8-59dd-46d3-8880-9851cd503ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:22.092883 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec546d8-59dd-46d3-8880-9851cd503ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:22.093171 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec546d8-59dd-46d3-8880-9851cd503ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:22.093453 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec546d8-59dd-46d3-8880-9851cd503ca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:22.094177 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec546d8-59dd-46d3-8880-9851cd503ca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:22.095799 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec546d8-59dd-46d3-8880-9851cd503ca3 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:22.106674 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1076/2134] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:18:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:18:22.110711 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0944785-cfce-4005-818d-7811e00cd67b admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:22.120695 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0944785-cfce-4005-818d-7811e00cd67b admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:22.125755 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e72a6-399b-4d3d-b1d4-32dc6dfc7d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:22.126641 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e72a6-399b-4d3d-b1d4-32dc6dfc7d63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:22.126928 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e72a6-399b-4d3d-b1d4-32dc6dfc7d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:22.127205 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e72a6-399b-4d3d-b1d4-32dc6dfc7d63 admin admin] PATH_INFO: `/v3/projects/7d586c0c8dd34398af9d52425eb2bde3/users/331c97a12f1b4c2eb1eefa61056ce089/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:22.144756 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e72a6-399b-4d3d-b1d4-32dc6dfc7d63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d586c0c8dd34398af9d52425eb2bde3, user_id=331c97a12f1b4c2eb1eefa61056ce089, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:22.148373 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e72a6-399b-4d3d-b1d4-32dc6dfc7d63 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:22.162536 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1059/2135] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:18:22 2022] PUT /identity/v3/projects/7d586c0c8dd34398af9d52425eb2bde3/users/331c97a12f1b4c2eb1eefa61056ce089/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:18:22.167279 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc77e11-4b7f-4bca-b398-fd12fd1d4e95 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:22.167609 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc77e11-4b7f-4bca-b398-fd12fd1d4e95 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:22.167893 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc77e11-4b7f-4bca-b398-fd12fd1d4e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:22.186843 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-3fc77e11-4b7f-4bca-b398-fd12fd1d4e95 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:22.188219 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-3fc77e11-4b7f-4bca-b398-fd12fd1d4e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:22.251325 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1077/2136] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:22.260927 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e72a6-399b-4d3d-b1d4-32dc6dfc7d63 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:22.262679 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-735e72a6-399b-4d3d-b1d4-32dc6dfc7d63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:22.317922 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e72a6-399b-4d3d-b1d4-32dc6dfc7d63 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:22.321579 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384bccc4-66c1-43d5-901c-7a0b60214381 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:22.322153 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384bccc4-66c1-43d5-901c-7a0b60214381 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:22.322438 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384bccc4-66c1-43d5-901c-7a0b60214381 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:22.322714 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384bccc4-66c1-43d5-901c-7a0b60214381 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:22.331910 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384bccc4-66c1-43d5-901c-7a0b60214381 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:22.333330 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384bccc4-66c1-43d5-901c-7a0b60214381 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:22.342587 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1060/2137] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:18:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:18:26.758503 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405d7fb-fc6f-4bcf-99f5-b55e83b0deef None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:26.759002 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405d7fb-fc6f-4bcf-99f5-b55e83b0deef None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:26.759104 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405d7fb-fc6f-4bcf-99f5-b55e83b0deef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:26.793352 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-e405d7fb-fc6f-4bcf-99f5-b55e83b0deef None None] MFA Rules not processed for user `331c97a12f1b4c2eb1eefa61056ce089`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:26.795090 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-e405d7fb-fc6f-4bcf-99f5-b55e83b0deef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:26.800994 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:26.863438 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1078/2138] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:26.867794 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384bccc4-66c1-43d5-901c-7a0b60214381 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:26.877708 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384bccc4-66c1-43d5-901c-7a0b60214381 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:26.882301 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2097eed-19d8-4485-83f2-2db9cc9c8b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:26.882959 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2097eed-19d8-4485-83f2-2db9cc9c8b92 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:26.883242 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2097eed-19d8-4485-83f2-2db9cc9c8b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:26.883586 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2097eed-19d8-4485-83f2-2db9cc9c8b92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:26.884276 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2097eed-19d8-4485-83f2-2db9cc9c8b92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:26.886186 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2097eed-19d8-4485-83f2-2db9cc9c8b92 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:26.918231 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1061/2139] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:18:26 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:18:26.919056 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e405d7fb-fc6f-4bcf-99f5-b55e83b0deef None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:26.932234 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e405d7fb-fc6f-4bcf-99f5-b55e83b0deef None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:26.994653 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6326111c-e3fb-48af-bbab-5b1c8ed01c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:26.995372 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326111c-e3fb-48af-bbab-5b1c8ed01c7b admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:26.995662 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326111c-e3fb-48af-bbab-5b1c8ed01c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:26.995942 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326111c-e3fb-48af-bbab-5b1c8ed01c7b admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:26.996665 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6326111c-e3fb-48af-bbab-5b1c8ed01c7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:26.998598 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6326111c-e3fb-48af-bbab-5b1c8ed01c7b admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:27.020760 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1079/2140] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:18:26 2022] POST /identity/v3/users => generated 295 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:18:27.024678 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2097eed-19d8-4485-83f2-2db9cc9c8b92 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:27.040033 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2097eed-19d8-4485-83f2-2db9cc9c8b92 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:27.049507 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336ae2f-05ff-461e-8620-4d9daa0b72f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:27.050694 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336ae2f-05ff-461e-8620-4d9daa0b72f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:27.050694 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336ae2f-05ff-461e-8620-4d9daa0b72f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:27.050905 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336ae2f-05ff-461e-8620-4d9daa0b72f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:27.052038 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336ae2f-05ff-461e-8620-4d9daa0b72f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:27.054008 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336ae2f-05ff-461e-8620-4d9daa0b72f7 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:27.068263 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1062/2141] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:18:27.072724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6326111c-e3fb-48af-bbab-5b1c8ed01c7b admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:27.094745 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6326111c-e3fb-48af-bbab-5b1c8ed01c7b admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:27.102742 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c8a7c4-c8a5-4dae-b9b1-5ba9e1b4a5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:27.103705 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8a7c4-c8a5-4dae-b9b1-5ba9e1b4a5ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:27.103985 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8a7c4-c8a5-4dae-b9b1-5ba9e1b4a5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:27.104257 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8a7c4-c8a5-4dae-b9b1-5ba9e1b4a5ad admin admin] PATH_INFO: `/v3/projects/d3190959220e4ae0be2e9d8e696477ba/users/31fad66b7380445a8ba7e13c6d443df0/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:27.122647 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c8a7c4-c8a5-4dae-b9b1-5ba9e1b4a5ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3190959220e4ae0be2e9d8e696477ba, user_id=31fad66b7380445a8ba7e13c6d443df0, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:27.125418 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c8a7c4-c8a5-4dae-b9b1-5ba9e1b4a5ad admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:27.140475 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1080/2142] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:18:27 2022] PUT /identity/v3/projects/d3190959220e4ae0be2e9d8e696477ba/users/31fad66b7380445a8ba7e13c6d443df0/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:18:27.144393 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336ae2f-05ff-461e-8620-4d9daa0b72f7 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:27.154598 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336ae2f-05ff-461e-8620-4d9daa0b72f7 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:27.191652 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:27.217869 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ae3a6-7078-4c6a-8b78-c76da89113e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:27.218567 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ae3a6-7078-4c6a-8b78-c76da89113e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:27.218846 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ae3a6-7078-4c6a-8b78-c76da89113e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:27.219122 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ae3a6-7078-4c6a-8b78-c76da89113e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:27.219877 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742ae3a6-7078-4c6a-8b78-c76da89113e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:27.221469 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742ae3a6-7078-4c6a-8b78-c76da89113e4 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:27.235963 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1063/2143] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:18:27.240024 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c8a7c4-c8a5-4dae-b9b1-5ba9e1b4a5ad admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:27.258438 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c8a7c4-c8a5-4dae-b9b1-5ba9e1b4a5ad admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:27.266371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd983bc9-1329-408d-9468-1979d0cb2239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:27.267385 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd983bc9-1329-408d-9468-1979d0cb2239 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:27.267737 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd983bc9-1329-408d-9468-1979d0cb2239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:27.268086 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd983bc9-1329-408d-9468-1979d0cb2239 admin admin] PATH_INFO: `/v3/projects/d3190959220e4ae0be2e9d8e696477ba/users/31fad66b7380445a8ba7e13c6d443df0/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:27.270280 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd983bc9-1329-408d-9468-1979d0cb2239 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3190959220e4ae0be2e9d8e696477ba, user_id=31fad66b7380445a8ba7e13c6d443df0, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:27.273027 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd983bc9-1329-408d-9468-1979d0cb2239 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:27.284248 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1081/2144] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:18:27 2022] PUT /identity/v3/projects/d3190959220e4ae0be2e9d8e696477ba/users/31fad66b7380445a8ba7e13c6d443df0/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:18:32.046745 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f7e01-06fd-47bd-8d20-7ab362e75aa8 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:32.047343 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f7e01-06fd-47bd-8d20-7ab362e75aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:32.047343 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f7e01-06fd-47bd-8d20-7ab362e75aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:32.078040 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-dd5f7e01-06fd-47bd-8d20-7ab362e75aa8 None None] MFA Rules not processed for user `31fad66b7380445a8ba7e13c6d443df0`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:32.080047 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-dd5f7e01-06fd-47bd-8d20-7ab362e75aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:32.165477 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1064/2145] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:32 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:32.180261 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd983bc9-1329-408d-9468-1979d0cb2239 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:32.182720 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-fd983bc9-1329-408d-9468-1979d0cb2239 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:32.250602 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:32.296135 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd983bc9-1329-408d-9468-1979d0cb2239 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:32.302237 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d1c3e-acff-4da8-ba58-d99199d85f37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:32.302907 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d1c3e-acff-4da8-ba58-d99199d85f37 service nova] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:32.303191 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d1c3e-acff-4da8-ba58-d99199d85f37 service nova] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:32.303510 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d1c3e-acff-4da8-ba58-d99199d85f37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:32.371941 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d1c3e-acff-4da8-ba58-d99199d85f37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:32.373545 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d1c3e-acff-4da8-ba58-d99199d85f37 service nova] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:32.383319 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1082/2146] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:18:32 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:18:41.759769 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f7e01-06fd-47bd-8d20-7ab362e75aa8 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:41.761323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-dd5f7e01-06fd-47bd-8d20-7ab362e75aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:41.834265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f7e01-06fd-47bd-8d20-7ab362e75aa8 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:41.838865 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b9f03-c95e-439a-bf08-6ce8cc2a4c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:41.839776 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b9f03-c95e-439a-bf08-6ce8cc2a4c26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:41.840058 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b9f03-c95e-439a-bf08-6ce8cc2a4c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:41.840329 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b9f03-c95e-439a-bf08-6ce8cc2a4c26 admin admin] PATH_INFO: `/v3/users/42cbcce54d6b4da3a0bffa1b923a605a` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:41.841507 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355b9f03-c95e-439a-bf08-6ce8cc2a4c26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42cbcce54d6b4da3a0bffa1b923a605a)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:41.843516 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355b9f03-c95e-439a-bf08-6ce8cc2a4c26 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:41.889420 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-355b9f03-c95e-439a-bf08-6ce8cc2a4c26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42cbcce54d6b4da3a0bffa1b923a605a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:41.897265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-355b9f03-c95e-439a-bf08-6ce8cc2a4c26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42cbcce54d6b4da3a0bffa1b923a605a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:41.902240 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:41.912112 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-355b9f03-c95e-439a-bf08-6ce8cc2a4c26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42cbcce54d6b4da3a0bffa1b923a605a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:41.912970 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-355b9f03-c95e-439a-bf08-6ce8cc2a4c26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42cbcce54d6b4da3a0bffa1b923a605a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:41.920354 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-355b9f03-c95e-439a-bf08-6ce8cc2a4c26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42cbcce54d6b4da3a0bffa1b923a605a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:41.926128 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1065/2147] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:18:41 2022] DELETE /identity/v3/users/42cbcce54d6b4da3a0bffa1b923a605a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:18:41.930734 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eaf62e-5ee0-412e-9e6f-3a8dd962cb22 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:41.931273 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eaf62e-5ee0-412e-9e6f-3a8dd962cb22 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:41.931273 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eaf62e-5ee0-412e-9e6f-3a8dd962cb22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:41.950913 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-35eaf62e-5ee0-412e-9e6f-3a8dd962cb22 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:41.952146 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-35eaf62e-5ee0-412e-9e6f-3a8dd962cb22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:42.012099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1083/2148] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:42.021736 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b9f03-c95e-439a-bf08-6ce8cc2a4c26 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:42.023121 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-355b9f03-c95e-439a-bf08-6ce8cc2a4c26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:42.082347 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b9f03-c95e-439a-bf08-6ce8cc2a4c26 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:42.086014 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa9cd6c-e4e4-4513-8b71-15421033f699 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:42.086664 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9cd6c-e4e4-4513-8b71-15421033f699 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:42.086949 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9cd6c-e4e4-4513-8b71-15421033f699 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:42.087225 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9cd6c-e4e4-4513-8b71-15421033f699 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:42.097208 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa9cd6c-e4e4-4513-8b71-15421033f699 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:42.098637 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa9cd6c-e4e4-4513-8b71-15421033f699 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:42.108145 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1066/2149] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:18:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:18:42.385689 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eaf62e-5ee0-412e-9e6f-3a8dd962cb22 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:42.387804 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-35eaf62e-5ee0-412e-9e6f-3a8dd962cb22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:42.399644 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eaf62e-5ee0-412e-9e6f-3a8dd962cb22 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:42.405443 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe252cb-9032-48bd-899f-d1c0171c75e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:42.406278 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe252cb-9032-48bd-899f-d1c0171c75e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:42.406566 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe252cb-9032-48bd-899f-d1c0171c75e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:42.407438 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe252cb-9032-48bd-899f-d1c0171c75e3 admin admin] PATH_INFO: `/v3/projects/9aa2047f5cde4b5b930b2518105705de` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:42.408824 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe252cb-9032-48bd-899f-d1c0171c75e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9aa2047f5cde4b5b930b2518105705de)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:42.410738 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe252cb-9032-48bd-899f-d1c0171c75e3 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:42.459622 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0fe252cb-9032-48bd-899f-d1c0171c75e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9aa2047f5cde4b5b930b2518105705de', 'request_id': 'req-0fe252cb-9032-48bd-899f-d1c0171c75e3'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:42.465795 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0fe252cb-9032-48bd-899f-d1c0171c75e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9aa2047f5cde4b5b930b2518105705de', 'request_id': 'req-0fe252cb-9032-48bd-899f-d1c0171c75e3'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:42.473083 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0fe252cb-9032-48bd-899f-d1c0171c75e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9aa2047f5cde4b5b930b2518105705de', 'request_id': 'req-0fe252cb-9032-48bd-899f-d1c0171c75e3'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:42.477243 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:42.485837 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0fe252cb-9032-48bd-899f-d1c0171c75e3 admin admin] The token cache is being invalidate because project 9aa2047f5cde4b5b930b2518105705de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:18:42.486329 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0fe252cb-9032-48bd-899f-d1c0171c75e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:42.487521 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0fe252cb-9032-48bd-899f-d1c0171c75e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:42.489233 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1084/2150] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:18:42 2022] DELETE /identity/v3/projects/9aa2047f5cde4b5b930b2518105705de => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:18:42.494083 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef640b2-507b-4705-938c-b7fd1d227b02 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:42.494393 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef640b2-507b-4705-938c-b7fd1d227b02 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:42.494692 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef640b2-507b-4705-938c-b7fd1d227b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:42.513306 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-2ef640b2-507b-4705-938c-b7fd1d227b02 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:42.514510 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2ef640b2-507b-4705-938c-b7fd1d227b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:42.574437 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1067/2151] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:42.583068 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb4639-0f85-46a2-b30d-0f6a299ac93e None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:42.583389 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb4639-0f85-46a2-b30d-0f6a299ac93e None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:42.583676 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb4639-0f85-46a2-b30d-0f6a299ac93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:42.602054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-22eb4639-0f85-46a2-b30d-0f6a299ac93e None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:42.603255 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-22eb4639-0f85-46a2-b30d-0f6a299ac93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:42.612435 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1085/2152] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:42.616429 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef640b2-507b-4705-938c-b7fd1d227b02 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:42.626957 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef640b2-507b-4705-938c-b7fd1d227b02 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:42.631594 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a13e9-4a83-4bba-a833-37b19d2fc218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:42.632259 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a13e9-4a83-4bba-a833-37b19d2fc218 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:42.632545 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a13e9-4a83-4bba-a833-37b19d2fc218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:42.632826 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a13e9-4a83-4bba-a833-37b19d2fc218 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:42.633562 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a13e9-4a83-4bba-a833-37b19d2fc218 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:18:42.633875 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a13e9-4a83-4bba-a833-37b19d2fc218 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:42.635495 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a13e9-4a83-4bba-a833-37b19d2fc218 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:42.651463 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1068/2153] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:18:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:18:42.656798 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb4639-0f85-46a2-b30d-0f6a299ac93e None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:42.666304 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb4639-0f85-46a2-b30d-0f6a299ac93e None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:42.672510 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8910c-c2c7-4697-ba89-334455e90c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:42.673196 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8910c-c2c7-4697-ba89-334455e90c28 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:42.673467 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8910c-c2c7-4697-ba89-334455e90c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:42.673746 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8910c-c2c7-4697-ba89-334455e90c28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:42.674434 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d8910c-c2c7-4697-ba89-334455e90c28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:42.676369 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d8910c-c2c7-4697-ba89-334455e90c28 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:42.705651 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1086/2154] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:18:42 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:18:42.710057 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a13e9-4a83-4bba-a833-37b19d2fc218 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:42.722190 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a13e9-4a83-4bba-a833-37b19d2fc218 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:42.779056 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37312539-a2bd-4b7a-904e-2fcab09f8d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:42.779772 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37312539-a2bd-4b7a-904e-2fcab09f8d4e admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:42.780022 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37312539-a2bd-4b7a-904e-2fcab09f8d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:42.780298 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37312539-a2bd-4b7a-904e-2fcab09f8d4e admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:42.780990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37312539-a2bd-4b7a-904e-2fcab09f8d4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:42.782891 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37312539-a2bd-4b7a-904e-2fcab09f8d4e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:42.796660 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:42.807617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1069/2155] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:18:42 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:18:42.811414 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8910c-c2c7-4697-ba89-334455e90c28 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:42.821561 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8910c-c2c7-4697-ba89-334455e90c28 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:42.826234 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88394374-33cb-4d22-98a5-c8207a58977a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:42.826886 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88394374-33cb-4d22-98a5-c8207a58977a admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:42.827173 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88394374-33cb-4d22-98a5-c8207a58977a admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:42.827490 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88394374-33cb-4d22-98a5-c8207a58977a admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:42.828332 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88394374-33cb-4d22-98a5-c8207a58977a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:42.829997 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88394374-33cb-4d22-98a5-c8207a58977a admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:42.841485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1087/2156] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:18:42.845227 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37312539-a2bd-4b7a-904e-2fcab09f8d4e admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:42.854820 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37312539-a2bd-4b7a-904e-2fcab09f8d4e admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:42.860452 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c70a0-a2b5-458e-9cd3-10d0b0004286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:42.861147 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c70a0-a2b5-458e-9cd3-10d0b0004286 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:42.861435 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c70a0-a2b5-458e-9cd3-10d0b0004286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:42.861760 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c70a0-a2b5-458e-9cd3-10d0b0004286 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:42.862547 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c70a0-a2b5-458e-9cd3-10d0b0004286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:42.864202 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c70a0-a2b5-458e-9cd3-10d0b0004286 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:42.877888 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1070/2157] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:18:42.878945 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88394374-33cb-4d22-98a5-c8207a58977a admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:42.888801 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88394374-33cb-4d22-98a5-c8207a58977a admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:42.894045 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8b04b-b419-4645-bf23-e14e5603f914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:42.894963 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8b04b-b419-4645-bf23-e14e5603f914 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:42.895243 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8b04b-b419-4645-bf23-e14e5603f914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:42.895578 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8b04b-b419-4645-bf23-e14e5603f914 admin admin] PATH_INFO: `/v3/projects/af4ab08d1c85451f849bb24ed4b18ac6/users/36cae367ad444cd6b2d71105b56dab9b/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:42.911996 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac8b04b-b419-4645-bf23-e14e5603f914 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af4ab08d1c85451f849bb24ed4b18ac6, user_id=36cae367ad444cd6b2d71105b56dab9b, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:42.914661 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac8b04b-b419-4645-bf23-e14e5603f914 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:42.931612 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1088/2158] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:18:42 2022] PUT /identity/v3/projects/af4ab08d1c85451f849bb24ed4b18ac6/users/36cae367ad444cd6b2d71105b56dab9b/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:18:42.941843 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde27e9-a409-4eb0-a337-9ded2395ee43 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:42.942180 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde27e9-a409-4eb0-a337-9ded2395ee43 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:42.942414 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde27e9-a409-4eb0-a337-9ded2395ee43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:42.973123 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-3fde27e9-a409-4eb0-a337-9ded2395ee43 None None] MFA Rules not processed for user `36cae367ad444cd6b2d71105b56dab9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:42.974361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3fde27e9-a409-4eb0-a337-9ded2395ee43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:43.061986 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1071/2159] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:43.071080 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8b04b-b419-4645-bf23-e14e5603f914 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:43.072829 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-1ac8b04b-b419-4645-bf23-e14e5603f914 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:43.175315 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8b04b-b419-4645-bf23-e14e5603f914 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:43.180192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbe36e3-dfd5-46d8-87f2-64d351daed00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:43.180919 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe36e3-dfd5-46d8-87f2-64d351daed00 service cinder] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:43.181188 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe36e3-dfd5-46d8-87f2-64d351daed00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:43.181549 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe36e3-dfd5-46d8-87f2-64d351daed00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:43.191922 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbe36e3-dfd5-46d8-87f2-64d351daed00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:43.193366 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbe36e3-dfd5-46d8-87f2-64d351daed00 service cinder] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:43.204307 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1089/2160] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:18:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:18:43.235458 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde27e9-a409-4eb0-a337-9ded2395ee43 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:43.340842 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde27e9-a409-4eb0-a337-9ded2395ee43 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:43.346663 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58387b-8e17-484e-b32d-4e208c8a3b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:43.347588 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58387b-8e17-484e-b32d-4e208c8a3b74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:43.347876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58387b-8e17-484e-b32d-4e208c8a3b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:43.348158 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58387b-8e17-484e-b32d-4e208c8a3b74 admin admin] PATH_INFO: `/v3/users/36cae367ad444cd6b2d71105b56dab9b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:43.349421 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58387b-8e17-484e-b32d-4e208c8a3b74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36cae367ad444cd6b2d71105b56dab9b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:43.351478 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58387b-8e17-484e-b32d-4e208c8a3b74 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:43.400103 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0c58387b-8e17-484e-b32d-4e208c8a3b74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36cae367ad444cd6b2d71105b56dab9b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:43.408386 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0c58387b-8e17-484e-b32d-4e208c8a3b74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36cae367ad444cd6b2d71105b56dab9b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:43.421269 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0c58387b-8e17-484e-b32d-4e208c8a3b74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36cae367ad444cd6b2d71105b56dab9b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:43.421967 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0c58387b-8e17-484e-b32d-4e208c8a3b74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36cae367ad444cd6b2d71105b56dab9b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:43.426391 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:43.430915 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0c58387b-8e17-484e-b32d-4e208c8a3b74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36cae367ad444cd6b2d71105b56dab9b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:43.438493 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1072/2161] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:18:43 2022] DELETE /identity/v3/users/36cae367ad444cd6b2d71105b56dab9b => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:18:43.443024 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555eadc-9824-469d-9515-0783ca03f384 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:43.443606 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555eadc-9824-469d-9515-0783ca03f384 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:43.443993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555eadc-9824-469d-9515-0783ca03f384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:43.463404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-6555eadc-9824-469d-9515-0783ca03f384 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:43.465554 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-6555eadc-9824-469d-9515-0783ca03f384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:43.526519 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1090/2162] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:43.536979 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58387b-8e17-484e-b32d-4e208c8a3b74 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:43.538941 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0c58387b-8e17-484e-b32d-4e208c8a3b74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:43.606768 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58387b-8e17-484e-b32d-4e208c8a3b74 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:43.610773 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ecdae-7c30-4735-a39c-5df09aa30479 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:43.611494 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ecdae-7c30-4735-a39c-5df09aa30479 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:43.611855 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ecdae-7c30-4735-a39c-5df09aa30479 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:43.612081 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ecdae-7c30-4735-a39c-5df09aa30479 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:43.625441 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2ecdae-7c30-4735-a39c-5df09aa30479 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:43.626887 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2ecdae-7c30-4735-a39c-5df09aa30479 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:43.637570 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1073/2163] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:18:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:18:43.918974 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555eadc-9824-469d-9515-0783ca03f384 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:43.920534 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-6555eadc-9824-469d-9515-0783ca03f384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:43.932842 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555eadc-9824-469d-9515-0783ca03f384 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:43.938549 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e84aaa-6d0c-4d3f-9e1f-6d5212537d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:43.939537 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e84aaa-6d0c-4d3f-9e1f-6d5212537d6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:43.939898 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e84aaa-6d0c-4d3f-9e1f-6d5212537d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:43.940264 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e84aaa-6d0c-4d3f-9e1f-6d5212537d6e admin admin] PATH_INFO: `/v3/projects/af4ab08d1c85451f849bb24ed4b18ac6` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:43.941599 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e84aaa-6d0c-4d3f-9e1f-6d5212537d6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=af4ab08d1c85451f849bb24ed4b18ac6)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:43.943591 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e84aaa-6d0c-4d3f-9e1f-6d5212537d6e admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:43.953041 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:44.003096 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-c6e84aaa-6d0c-4d3f-9e1f-6d5212537d6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af4ab08d1c85451f849bb24ed4b18ac6', 'request_id': 'req-c6e84aaa-6d0c-4d3f-9e1f-6d5212537d6e'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:44.010068 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-c6e84aaa-6d0c-4d3f-9e1f-6d5212537d6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af4ab08d1c85451f849bb24ed4b18ac6', 'request_id': 'req-c6e84aaa-6d0c-4d3f-9e1f-6d5212537d6e'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:44.018285 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-c6e84aaa-6d0c-4d3f-9e1f-6d5212537d6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af4ab08d1c85451f849bb24ed4b18ac6', 'request_id': 'req-c6e84aaa-6d0c-4d3f-9e1f-6d5212537d6e'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:44.030119 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-c6e84aaa-6d0c-4d3f-9e1f-6d5212537d6e admin admin] The token cache is being invalidate because project af4ab08d1c85451f849bb24ed4b18ac6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:18:44.030119 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-c6e84aaa-6d0c-4d3f-9e1f-6d5212537d6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:44.030388 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-c6e84aaa-6d0c-4d3f-9e1f-6d5212537d6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:44.032046 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1091/2164] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:18:43 2022] DELETE /identity/v3/projects/af4ab08d1c85451f849bb24ed4b18ac6 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:18:44.037063 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90a8fb-cd89-4b1e-84a8-3a5502c9eea3 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:44.037538 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90a8fb-cd89-4b1e-84a8-3a5502c9eea3 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:44.038019 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90a8fb-cd89-4b1e-84a8-3a5502c9eea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:44.061188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-ee90a8fb-cd89-4b1e-84a8-3a5502c9eea3 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:44.062510 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ee90a8fb-cd89-4b1e-84a8-3a5502c9eea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:44.124232 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1074/2165] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:18:44.131697 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfeb486-0731-4eb7-afa0-d23f2eade562 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:44.132104 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfeb486-0731-4eb7-afa0-d23f2eade562 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:44.132470 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfeb486-0731-4eb7-afa0-d23f2eade562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:44.151161 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-ecfeb486-0731-4eb7-afa0-d23f2eade562 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:44.152425 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-ecfeb486-0731-4eb7-afa0-d23f2eade562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:44.160528 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1092/2166] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:18:44.165204 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90a8fb-cd89-4b1e-84a8-3a5502c9eea3 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:44.176330 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90a8fb-cd89-4b1e-84a8-3a5502c9eea3 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:44.181227 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e253f-6be1-4e46-8c96-53cb169adfe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:44.182038 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e253f-6be1-4e46-8c96-53cb169adfe1 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:44.182403 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e253f-6be1-4e46-8c96-53cb169adfe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:44.182720 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e253f-6be1-4e46-8c96-53cb169adfe1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:44.184103 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711e253f-6be1-4e46-8c96-53cb169adfe1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:18:44.185010 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711e253f-6be1-4e46-8c96-53cb169adfe1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:44.186213 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711e253f-6be1-4e46-8c96-53cb169adfe1 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:44.198160 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1075/2167] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:18:44.202735 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfeb486-0731-4eb7-afa0-d23f2eade562 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:44.212305 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfeb486-0731-4eb7-afa0-d23f2eade562 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:44.217110 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3936a49-54ad-44d9-bad9-0a3adb968682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:44.217806 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3936a49-54ad-44d9-bad9-0a3adb968682 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:44.218096 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3936a49-54ad-44d9-bad9-0a3adb968682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:44.218375 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3936a49-54ad-44d9-bad9-0a3adb968682 admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:44.219080 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3936a49-54ad-44d9-bad9-0a3adb968682 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:44.221051 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3936a49-54ad-44d9-bad9-0a3adb968682 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:44.251575 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1093/2168] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:18:44 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:18:44.257150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e253f-6be1-4e46-8c96-53cb169adfe1 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:44.267475 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e253f-6be1-4e46-8c96-53cb169adfe1 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:44.323926 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b4983-54f4-4329-aca6-e28fd964082e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:44.324590 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b4983-54f4-4329-aca6-e28fd964082e admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:44.324872 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b4983-54f4-4329-aca6-e28fd964082e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:44.325143 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b4983-54f4-4329-aca6-e28fd964082e admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:44.325827 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b4983-54f4-4329-aca6-e28fd964082e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:44.327774 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b4983-54f4-4329-aca6-e28fd964082e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:44.348257 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1076/2169] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:18:44 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:18:44.352038 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3936a49-54ad-44d9-bad9-0a3adb968682 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:44.363405 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3936a49-54ad-44d9-bad9-0a3adb968682 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:44.370856 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad25fb-570f-4dd3-a164-9ff48f83f229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:44.372661 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad25fb-570f-4dd3-a164-9ff48f83f229 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:44.374486 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad25fb-570f-4dd3-a164-9ff48f83f229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:44.374874 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad25fb-570f-4dd3-a164-9ff48f83f229 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:44.375762 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbad25fb-570f-4dd3-a164-9ff48f83f229 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:44.378532 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbad25fb-570f-4dd3-a164-9ff48f83f229 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:44.390133 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1094/2170] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:18:44.398464 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b4983-54f4-4329-aca6-e28fd964082e admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:44.411524 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b4983-54f4-4329-aca6-e28fd964082e admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:44.416507 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4430ebe1-be37-49ad-934d-82a3038b23ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:44.417122 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430ebe1-be37-49ad-934d-82a3038b23ec admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:44.417404 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430ebe1-be37-49ad-934d-82a3038b23ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:44.418323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430ebe1-be37-49ad-934d-82a3038b23ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:44.419057 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4430ebe1-be37-49ad-934d-82a3038b23ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:44.420691 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4430ebe1-be37-49ad-934d-82a3038b23ec admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:44.430911 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1077/2171] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:18:44.435445 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad25fb-570f-4dd3-a164-9ff48f83f229 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:44.445883 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad25fb-570f-4dd3-a164-9ff48f83f229 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:44.452180 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0953d86b-6c77-49c7-93d7-e2e28b483674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:44.453092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0953d86b-6c77-49c7-93d7-e2e28b483674 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:44.453374 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0953d86b-6c77-49c7-93d7-e2e28b483674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:44.453648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0953d86b-6c77-49c7-93d7-e2e28b483674 admin admin] PATH_INFO: `/v3/projects/1b5c201aae5e438c81d8976289ffb4cb/users/12a595db7f394c7e86b6a9e0c9f625f8/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:44.469421 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0953d86b-6c77-49c7-93d7-e2e28b483674 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b5c201aae5e438c81d8976289ffb4cb, user_id=12a595db7f394c7e86b6a9e0c9f625f8, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:44.472094 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0953d86b-6c77-49c7-93d7-e2e28b483674 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:44.487895 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1095/2172] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:18:44 2022] PUT /identity/v3/projects/1b5c201aae5e438c81d8976289ffb4cb/users/12a595db7f394c7e86b6a9e0c9f625f8/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:18:44.493197 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e1f6d-282a-4f32-b47d-a40591cb9372 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:44.493598 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e1f6d-282a-4f32-b47d-a40591cb9372 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:44.493882 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e1f6d-282a-4f32-b47d-a40591cb9372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:44.499936 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:44.513979 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-6c1e1f6d-282a-4f32-b47d-a40591cb9372 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:44.515261 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6c1e1f6d-282a-4f32-b47d-a40591cb9372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:44.574328 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1078/2173] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:44.584300 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0953d86b-6c77-49c7-93d7-e2e28b483674 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:44.585954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-0953d86b-6c77-49c7-93d7-e2e28b483674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:44.645343 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0953d86b-6c77-49c7-93d7-e2e28b483674 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:44.648928 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58271505-1fde-4497-a433-9a37c0748b42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:44.649574 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58271505-1fde-4497-a433-9a37c0748b42 service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:44.649854 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58271505-1fde-4497-a433-9a37c0748b42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:44.650127 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58271505-1fde-4497-a433-9a37c0748b42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:44.659722 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58271505-1fde-4497-a433-9a37c0748b42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:44.661136 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58271505-1fde-4497-a433-9a37c0748b42 service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:44.670919 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1096/2174] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:18:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:18:45.317939 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c5fac-fcfe-4e50-a255-b233d6507417 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:45.318452 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c5fac-fcfe-4e50-a255-b233d6507417 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:45.318576 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c5fac-fcfe-4e50-a255-b233d6507417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:45.347913 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-991c5fac-fcfe-4e50-a255-b233d6507417 None None] MFA Rules not processed for user `12a595db7f394c7e86b6a9e0c9f625f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:45.349142 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-991c5fac-fcfe-4e50-a255-b233d6507417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:45.428298 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1079/2175] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:18:45.439340 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58271505-1fde-4497-a433-9a37c0748b42 service neutron] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:45.441267 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-58271505-1fde-4497-a433-9a37c0748b42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:45.487092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:45.507808 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58271505-1fde-4497-a433-9a37c0748b42 service neutron] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:45.511885 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e0a769-d368-4762-bb35-c3661b85497f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:45.512535 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0a769-d368-4762-bb35-c3661b85497f service cinder] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:45.512816 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0a769-d368-4762-bb35-c3661b85497f service cinder] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:45.513109 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0a769-d368-4762-bb35-c3661b85497f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:45.521839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e0a769-d368-4762-bb35-c3661b85497f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:45.523174 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e0a769-d368-4762-bb35-c3661b85497f service cinder] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:45.538505 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1097/2176] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:18:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:18:47.955788 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937c16e-a294-49a3-b726-eae390620fe2 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:47.956298 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937c16e-a294-49a3-b726-eae390620fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:47.956475 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937c16e-a294-49a3-b726-eae390620fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:47.977020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b937c16e-a294-49a3-b726-eae390620fe2 None None] MFA Rules not processed for user `12a595db7f394c7e86b6a9e0c9f625f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:47.978601 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b937c16e-a294-49a3-b726-eae390620fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:47.986682 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1080/2177] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:48.207429 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e0a769-d368-4762-bb35-c3661b85497f service cinder] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:48.208860 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-28e0a769-d368-4762-bb35-c3661b85497f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:48.222760 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e0a769-d368-4762-bb35-c3661b85497f service cinder] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:48.234245 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89604370-5d34-4680-b0c6-f21796d6da4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:48.235129 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89604370-5d34-4680-b0c6-f21796d6da4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:48.235501 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89604370-5d34-4680-b0c6-f21796d6da4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:48.235782 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89604370-5d34-4680-b0c6-f21796d6da4f admin admin] PATH_INFO: `/v3/users/9d9b356f076b45aebccd20245c652437` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:48.236966 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89604370-5d34-4680-b0c6-f21796d6da4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d9b356f076b45aebccd20245c652437)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:48.238966 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89604370-5d34-4680-b0c6-f21796d6da4f admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:48.286873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-89604370-5d34-4680-b0c6-f21796d6da4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d9b356f076b45aebccd20245c652437'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:48.295454 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-89604370-5d34-4680-b0c6-f21796d6da4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d9b356f076b45aebccd20245c652437'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:48.307718 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-89604370-5d34-4680-b0c6-f21796d6da4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d9b356f076b45aebccd20245c652437'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:48.315207 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-89604370-5d34-4680-b0c6-f21796d6da4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d9b356f076b45aebccd20245c652437'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:48.315925 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-89604370-5d34-4680-b0c6-f21796d6da4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d9b356f076b45aebccd20245c652437'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:48.322043 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1098/2178] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:18:48 2022] DELETE /identity/v3/users/9d9b356f076b45aebccd20245c652437 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:18:48.327352 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd19a35-8f36-4088-8203-78a988c14a76 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:48.327810 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd19a35-8f36-4088-8203-78a988c14a76 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:48.328087 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd19a35-8f36-4088-8203-78a988c14a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:48.348338 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-dfd19a35-8f36-4088-8203-78a988c14a76 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:48.350186 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-dfd19a35-8f36-4088-8203-78a988c14a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:48.411911 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1081/2179] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:48.421841 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89604370-5d34-4680-b0c6-f21796d6da4f admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:48.423242 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-89604370-5d34-4680-b0c6-f21796d6da4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:48.484229 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89604370-5d34-4680-b0c6-f21796d6da4f admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:48.488330 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c944f46d-015c-443c-ad06-72d9905ee027 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:48.491489 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c944f46d-015c-443c-ad06-72d9905ee027 service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:48.491965 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c944f46d-015c-443c-ad06-72d9905ee027 service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:48.492422 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c944f46d-015c-443c-ad06-72d9905ee027 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:48.503397 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c944f46d-015c-443c-ad06-72d9905ee027 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:48.504838 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c944f46d-015c-443c-ad06-72d9905ee027 service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:48.516275 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1099/2180] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:18:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:18:48.800554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd19a35-8f36-4088-8203-78a988c14a76 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:48.802395 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-dfd19a35-8f36-4088-8203-78a988c14a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:48.829500 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd19a35-8f36-4088-8203-78a988c14a76 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:48.835695 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7421b-1f13-4d22-8a2d-e821075b8e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:48.836647 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7421b-1f13-4d22-8a2d-e821075b8e7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:48.837005 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7421b-1f13-4d22-8a2d-e821075b8e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:48.837354 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7421b-1f13-4d22-8a2d-e821075b8e7c admin admin] PATH_INFO: `/v3/projects/91ee9661fe3c4c64849af4d4acce90db` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:48.838597 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f7421b-1f13-4d22-8a2d-e821075b8e7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=91ee9661fe3c4c64849af4d4acce90db)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:48.840803 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f7421b-1f13-4d22-8a2d-e821075b8e7c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:48.870870 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:48.897763 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f7f7421b-1f13-4d22-8a2d-e821075b8e7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91ee9661fe3c4c64849af4d4acce90db', 'request_id': 'req-f7f7421b-1f13-4d22-8a2d-e821075b8e7c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:48.905392 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f7f7421b-1f13-4d22-8a2d-e821075b8e7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91ee9661fe3c4c64849af4d4acce90db', 'request_id': 'req-f7f7421b-1f13-4d22-8a2d-e821075b8e7c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:48.912986 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f7f7421b-1f13-4d22-8a2d-e821075b8e7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91ee9661fe3c4c64849af4d4acce90db', 'request_id': 'req-f7f7421b-1f13-4d22-8a2d-e821075b8e7c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:48.923625 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f7f7421b-1f13-4d22-8a2d-e821075b8e7c admin admin] The token cache is being invalidate because project 91ee9661fe3c4c64849af4d4acce90db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:18:48.923912 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f7f7421b-1f13-4d22-8a2d-e821075b8e7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:48.924535 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f7f7421b-1f13-4d22-8a2d-e821075b8e7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:18:48.926069 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1082/2181] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:18:48 2022] DELETE /identity/v3/projects/91ee9661fe3c4c64849af4d4acce90db => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:18:48.930823 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a7b81-d9da-47a9-9910-df73262e4f1a None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:48.931110 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a7b81-d9da-47a9-9910-df73262e4f1a None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:48.931410 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a7b81-d9da-47a9-9910-df73262e4f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:48.952174 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-b98a7b81-d9da-47a9-9910-df73262e4f1a None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:48.953381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-b98a7b81-d9da-47a9-9910-df73262e4f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:49.004507 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:49.019164 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1100/2182] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:18:49.028210 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7421b-1f13-4d22-8a2d-e821075b8e7c admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:49.028825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601bf13-74b8-4edc-aac7-8751720a1a45 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:49.029124 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601bf13-74b8-4edc-aac7-8751720a1a45 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:49.029408 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601bf13-74b8-4edc-aac7-8751720a1a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:49.030107 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f7f7421b-1f13-4d22-8a2d-e821075b8e7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:49.048074 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-0601bf13-74b8-4edc-aac7-8751720a1a45 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:49.049297 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-0601bf13-74b8-4edc-aac7-8751720a1a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:49.058457 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1101/2183] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:49.061960 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601bf13-74b8-4edc-aac7-8751720a1a45 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:49.073150 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601bf13-74b8-4edc-aac7-8751720a1a45 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:49.079275 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706a883-7c0d-43e2-830e-5cdb52447729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:49.080246 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706a883-7c0d-43e2-830e-5cdb52447729 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:49.080701 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706a883-7c0d-43e2-830e-5cdb52447729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:49.081233 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706a883-7c0d-43e2-830e-5cdb52447729 admin admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:49.082365 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d706a883-7c0d-43e2-830e-5cdb52447729 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:18:49.082796 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d706a883-7c0d-43e2-830e-5cdb52447729 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:49.084544 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d706a883-7c0d-43e2-830e-5cdb52447729 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:49.100985 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1102/2184] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:18:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:18:49.102533 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706a883-7c0d-43e2-830e-5cdb52447729 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:49.113443 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706a883-7c0d-43e2-830e-5cdb52447729 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:49.119042 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0558015b-1572-42f1-8a26-f41a6d606ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:49.119707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558015b-1572-42f1-8a26-f41a6d606ee9 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:49.119994 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558015b-1572-42f1-8a26-f41a6d606ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:49.120984 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558015b-1572-42f1-8a26-f41a6d606ee9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:49.121687 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0558015b-1572-42f1-8a26-f41a6d606ee9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:49.124160 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0558015b-1572-42f1-8a26-f41a6d606ee9 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:49.148363 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7421b-1f13-4d22-8a2d-e821075b8e7c admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:49.156699 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1103/2185] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:18:49 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:18:49.156862 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad76381-f521-417b-a018-2233ce12b57f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be901786f8934d559e0bac5d622f97cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:49.156862 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad76381-f521-417b-a018-2233ce12b57f service placement] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:49.156862 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad76381-f521-417b-a018-2233ce12b57f service placement] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:49.156862 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad76381-f521-417b-a018-2233ce12b57f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:49.157239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6ad76381-f521-417b-a018-2233ce12b57f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:49.158569 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0558015b-1572-42f1-8a26-f41a6d606ee9 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:49.169537 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad76381-f521-417b-a018-2233ce12b57f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:49.171399 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad76381-f521-417b-a018-2233ce12b57f service placement] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:49.181977 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1083/2186] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:18:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:18:49.226825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0558015b-1572-42f1-8a26-f41a6d606ee9 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:49.232089 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c198f688-dec4-49ed-996d-a63f77b7d992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:49.232764 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c198f688-dec4-49ed-996d-a63f77b7d992 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:49.233048 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c198f688-dec4-49ed-996d-a63f77b7d992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:49.233334 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c198f688-dec4-49ed-996d-a63f77b7d992 admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:49.234024 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c198f688-dec4-49ed-996d-a63f77b7d992 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:49.235977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c198f688-dec4-49ed-996d-a63f77b7d992 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:49.264966 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1104/2187] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:18:49 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:18:49.271060 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad76381-f521-417b-a018-2233ce12b57f service placement] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:49.283716 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad76381-f521-417b-a018-2233ce12b57f service placement] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:49.288738 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3dcc05-baf0-4761-8280-000ee08ecb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:49.289586 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3dcc05-baf0-4761-8280-000ee08ecb24 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:49.289874 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3dcc05-baf0-4761-8280-000ee08ecb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:49.290683 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3dcc05-baf0-4761-8280-000ee08ecb24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:49.291945 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3dcc05-baf0-4761-8280-000ee08ecb24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:49.294740 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3dcc05-baf0-4761-8280-000ee08ecb24 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:49.311690 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1084/2188] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:18:49.316001 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c198f688-dec4-49ed-996d-a63f77b7d992 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:49.327434 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c198f688-dec4-49ed-996d-a63f77b7d992 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:49.332503 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2782aa5f-cf91-4d89-969d-62518c3144cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:49.333176 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782aa5f-cf91-4d89-969d-62518c3144cf admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:49.333484 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782aa5f-cf91-4d89-969d-62518c3144cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:49.333737 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782aa5f-cf91-4d89-969d-62518c3144cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:49.334501 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2782aa5f-cf91-4d89-969d-62518c3144cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:49.336271 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2782aa5f-cf91-4d89-969d-62518c3144cf admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:49.349955 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1105/2189] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:18:49.351265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3dcc05-baf0-4761-8280-000ee08ecb24 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:49.362561 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3dcc05-baf0-4761-8280-000ee08ecb24 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:49.368644 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058ecf4-f153-4be0-acd3-5a75f90c07d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:49.369591 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058ecf4-f153-4be0-acd3-5a75f90c07d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:49.369812 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058ecf4-f153-4be0-acd3-5a75f90c07d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:49.370088 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058ecf4-f153-4be0-acd3-5a75f90c07d2 admin admin] PATH_INFO: `/v3/projects/573ee03c74e3443d8a8d5e5568507d37/users/196748d4d1744c2d9fc441ea3a061dd9/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:49.386317 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058ecf4-f153-4be0-acd3-5a75f90c07d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=573ee03c74e3443d8a8d5e5568507d37, user_id=196748d4d1744c2d9fc441ea3a061dd9, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:49.399988 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058ecf4-f153-4be0-acd3-5a75f90c07d2 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:49.414628 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1085/2190] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:18:49 2022] PUT /identity/v3/projects/573ee03c74e3443d8a8d5e5568507d37/users/196748d4d1744c2d9fc441ea3a061dd9/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:18:49.420137 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2495d-c073-4d53-9446-98514a3edba4 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:49.420434 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2495d-c073-4d53-9446-98514a3edba4 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:49.420718 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2495d-c073-4d53-9446-98514a3edba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:49.446037 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-b9e2495d-c073-4d53-9446-98514a3edba4 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:49.447493 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-b9e2495d-c073-4d53-9446-98514a3edba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:49.535358 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1106/2191] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:49.545334 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058ecf4-f153-4be0-acd3-5a75f90c07d2 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:49.547123 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6058ecf4-f153-4be0-acd3-5a75f90c07d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:49.607391 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:49.625659 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058ecf4-f153-4be0-acd3-5a75f90c07d2 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:49.630658 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4032fc5-680b-43c8-b023-b5d5448d9424 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:49.631592 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4032fc5-680b-43c8-b023-b5d5448d9424 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:49.631850 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4032fc5-680b-43c8-b023-b5d5448d9424 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:49.632127 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4032fc5-680b-43c8-b023-b5d5448d9424 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:49.642820 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4032fc5-680b-43c8-b023-b5d5448d9424 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:49.644276 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4032fc5-680b-43c8-b023-b5d5448d9424 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:49.658577 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1086/2192] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:18:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:18:50.660057 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466e217-c591-4de3-9dc2-962602de92c3 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:50.660564 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466e217-c591-4de3-9dc2-962602de92c3 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:50.661067 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466e217-c591-4de3-9dc2-962602de92c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:50.717717 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-4466e217-c591-4de3-9dc2-962602de92c3 None None] MFA Rules not processed for user `196748d4d1744c2d9fc441ea3a061dd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:50.719098 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-4466e217-c591-4de3-9dc2-962602de92c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:50.764591 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:18:50.823331 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1107/2193] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:50.834253 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4032fc5-680b-43c8-b023-b5d5448d9424 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:18:50.836345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e4032fc5-680b-43c8-b023-b5d5448d9424 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:50.915450 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4032fc5-680b-43c8-b023-b5d5448d9424 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:18:50.919643 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce22ba-7495-4626-bffe-f8474c16aebd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:18:50.920294 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce22ba-7495-4626-bffe-f8474c16aebd service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:50.920579 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce22ba-7495-4626-bffe-f8474c16aebd service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:50.920856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce22ba-7495-4626-bffe-f8474c16aebd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:50.930111 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce22ba-7495-4626-bffe-f8474c16aebd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:18:50.931588 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce22ba-7495-4626-bffe-f8474c16aebd service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:18:50.941416 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1087/2194] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:18:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:18:53.305147 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c23dc-6a79-4d92-9a5b-24ee0a93f555 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:53.305625 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c23dc-6a79-4d92-9a5b-24ee0a93f555 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:53.305717 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c23dc-6a79-4d92-9a5b-24ee0a93f555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:53.324565 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-d65c23dc-6a79-4d92-9a5b-24ee0a93f555 None None] MFA Rules not processed for user `4206a21c317e4b5aace8541699f8032d`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:53.325768 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-d65c23dc-6a79-4d92-9a5b-24ee0a93f555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:53.357845 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98038e-d46b-4ea0-949e-ff25a40927d4 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:18:53.358339 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98038e-d46b-4ea0-949e-ff25a40927d4 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:18:53.358339 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98038e-d46b-4ea0-949e-ff25a40927d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:18:53.375800 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-be98038e-d46b-4ea0-949e-ff25a40927d4 None None] MFA Rules not processed for user `196748d4d1744c2d9fc441ea3a061dd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:18:53.376991 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-be98038e-d46b-4ea0-949e-ff25a40927d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:18:53.384477 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1108/2195] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:18:53.384650 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1088/2196] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:18:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:19:03.370584 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=35 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:19:03.374200 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=43 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:19:03.498832 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea930c8-cd40-4222-b956-80403c58cdca None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:19:03.499139 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea930c8-cd40-4222-b956-80403c58cdca None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:19:03.499438 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea930c8-cd40-4222-b956-80403c58cdca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:19:03.517563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-4ea930c8-cd40-4222-b956-80403c58cdca None None] MFA Rules not processed for user `331c97a12f1b4c2eb1eefa61056ce089`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:19:03.518873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-4ea930c8-cd40-4222-b956-80403c58cdca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:19:03.574072 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1109/2197] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:19:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:19:13.566323 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=25 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:19:21.174666 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b29db-c356-4649-bc34-49ec1c046d3c None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:19:21.175221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b29db-c356-4649-bc34-49ec1c046d3c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:19:21.175221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b29db-c356-4649-bc34-49ec1c046d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:19:21.194081 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-792b29db-c356-4649-bc34-49ec1c046d3c None None] MFA Rules not processed for user `196748d4d1744c2d9fc441ea3a061dd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:19:21.195403 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-792b29db-c356-4649-bc34-49ec1c046d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:19:21.202449 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1089/2198] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:19:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:19:31.188608 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:19:41.870075 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea930c8-cd40-4222-b956-80403c58cdca None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:19:41.879493 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea930c8-cd40-4222-b956-80403c58cdca None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:19:41.883051 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef5f2e-6a51-4104-a56f-4b1408fa7738 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:19:41.883720 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef5f2e-6a51-4104-a56f-4b1408fa7738 service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:19:41.884012 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef5f2e-6a51-4104-a56f-4b1408fa7738 service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:19:41.884291 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef5f2e-6a51-4104-a56f-4b1408fa7738 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:19:41.886025 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-f6ef5f2e-6a51-4104-a56f-4b1408fa7738 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:19:41.947838 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ef5f2e-6a51-4104-a56f-4b1408fa7738 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:19:41.949257 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ef5f2e-6a51-4104-a56f-4b1408fa7738 service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:19:41.959313 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1110/2199] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:19:41 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:19:51.955047 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=30 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:19:59.758572 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93183ca3-6bfb-4297-b939-de540ddfeb79 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:19:59.759073 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93183ca3-6bfb-4297-b939-de540ddfeb79 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:19:59.759166 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93183ca3-6bfb-4297-b939-de540ddfeb79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:19:59.806139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-93183ca3-6bfb-4297-b939-de540ddfeb79 None None] MFA Rules not processed for user `875140cf51d9410da45f8c3805b26204`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:19:59.807423 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-93183ca3-6bfb-4297-b939-de540ddfeb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:19:59.870753 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1090/2200] 10.222.0.41 () {58 vars in 1009 bytes} [Fri Jun 3 16:19:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:19:59.915527 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef5f2e-6a51-4104-a56f-4b1408fa7738 service neutron] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:19:59.916938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-f6ef5f2e-6a51-4104-a56f-4b1408fa7738 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:20:00.005738 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef5f2e-6a51-4104-a56f-4b1408fa7738 service neutron] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:20:00.010692 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746330d-3263-48c1-904c-268d533c1ca7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5f513b781a4a73b443cded13359eda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:20:00.011363 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746330d-3263-48c1-904c-268d533c1ca7 service swift] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:20:00.011655 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746330d-3263-48c1-904c-268d533c1ca7 service swift] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:20:00.012505 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746330d-3263-48c1-904c-268d533c1ca7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:20:00.020442 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5746330d-3263-48c1-904c-268d533c1ca7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:20:00.022020 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5746330d-3263-48c1-904c-268d533c1ca7 service swift] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:20:00.030236 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1111/2201] 10.222.0.41 () {60 vars in 1384 bytes} [Fri Jun 3 16:19:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 16:20:09.867850 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=40 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:20:10.027086 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=41 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:20:15.154874 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd31b2-18fa-406f-b7df-61fd1a7b2ecf None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:20:15.155480 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd31b2-18fa-406f-b7df-61fd1a7b2ecf None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:20:15.155480 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd31b2-18fa-406f-b7df-61fd1a7b2ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:20:15.173973 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-75dd31b2-18fa-406f-b7df-61fd1a7b2ecf None None] MFA Rules not processed for user `875140cf51d9410da45f8c3805b26204`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:20:15.175221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-75dd31b2-18fa-406f-b7df-61fd1a7b2ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:20:15.182420 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1091/2202] 10.222.0.41 () {58 vars in 967 bytes} [Fri Jun 3 16:20:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:20:15.194848 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746330d-3263-48c1-904c-268d533c1ca7 service swift] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:20:15.205480 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746330d-3263-48c1-904c-268d533c1ca7 service swift] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:20:15.210454 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1b847-e834-40ce-8454-0296a6ce086c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5f513b781a4a73b443cded13359eda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:20:15.211086 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1b847-e834-40ce-8454-0296a6ce086c service swift] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:20:15.211407 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1b847-e834-40ce-8454-0296a6ce086c service swift] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:20:15.211686 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1b847-e834-40ce-8454-0296a6ce086c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:20:15.220375 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e1b847-e834-40ce-8454-0296a6ce086c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:20:15.221973 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e1b847-e834-40ce-8454-0296a6ce086c service swift] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:20:15.230626 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1112/2203] 10.222.0.41 () {60 vars in 1384 bytes} [Fri Jun 3 16:20:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 16:20:17.185152 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a91f8-e565-4c7f-99ec-0a3a2da69d08 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:20:17.185669 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a91f8-e565-4c7f-99ec-0a3a2da69d08 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:20:17.185777 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a91f8-e565-4c7f-99ec-0a3a2da69d08 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:20:17.187086 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1092/2204] 10.222.0.41 () {54 vars in 921 bytes} [Fri Jun 3 16:20:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:20:17.222371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6feb1e5-a398-4a2b-955e-b9c827027f66 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:20:17.222840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6feb1e5-a398-4a2b-955e-b9c827027f66 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:20:17.222951 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6feb1e5-a398-4a2b-955e-b9c827027f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:20:17.264471 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a91f8-e565-4c7f-99ec-0a3a2da69d08 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:20:17.265137 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-c6feb1e5-a398-4a2b-955e-b9c827027f66 None None] MFA Rules not processed for user `be901786f8934d559e0bac5d622f97cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:20:17.266348 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-c6feb1e5-a398-4a2b-955e-b9c827027f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:20:17.278843 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a91f8-e565-4c7f-99ec-0a3a2da69d08 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:20:17.284135 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ccb20-50c5-411b-be91-26df9b34ae69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:20:17.285019 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ccb20-50c5-411b-be91-26df9b34ae69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:20:17.285304 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ccb20-50c5-411b-be91-26df9b34ae69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:20:17.285606 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ccb20-50c5-411b-be91-26df9b34ae69 admin admin] PATH_INFO: `/v3/users/196748d4d1744c2d9fc441ea3a061dd9` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:20:17.287392 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976ccb20-50c5-411b-be91-26df9b34ae69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=196748d4d1744c2d9fc441ea3a061dd9)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:20:17.289312 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976ccb20-50c5-411b-be91-26df9b34ae69 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:20:17.333681 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1113/2205] 10.222.0.41 () {58 vars in 1018 bytes} [Fri Jun 3 16:20:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:20:17.338307 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-976ccb20-50c5-411b-be91-26df9b34ae69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '196748d4d1744c2d9fc441ea3a061dd9'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:20:17.345308 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6feb1e5-a398-4a2b-955e-b9c827027f66 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:20:17.347002 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-976ccb20-50c5-411b-be91-26df9b34ae69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '196748d4d1744c2d9fc441ea3a061dd9'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:20:17.347526 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-c6feb1e5-a398-4a2b-955e-b9c827027f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:20:17.360253 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-976ccb20-50c5-411b-be91-26df9b34ae69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '196748d4d1744c2d9fc441ea3a061dd9'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:20:17.360590 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6feb1e5-a398-4a2b-955e-b9c827027f66 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:20:17.361192 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-976ccb20-50c5-411b-be91-26df9b34ae69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '196748d4d1744c2d9fc441ea3a061dd9'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:20:17.365601 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ebb62-8e8d-4b25-8748-eed12df0d917 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be901786f8934d559e0bac5d622f97cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:20:17.366244 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ebb62-8e8d-4b25-8748-eed12df0d917 service placement] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:20:17.366526 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ebb62-8e8d-4b25-8748-eed12df0d917 service placement] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:20:17.366833 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ebb62-8e8d-4b25-8748-eed12df0d917 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:20:17.368550 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-976ccb20-50c5-411b-be91-26df9b34ae69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '196748d4d1744c2d9fc441ea3a061dd9'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:20:17.368678 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-a44ebb62-8e8d-4b25-8748-eed12df0d917 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:20:17.374644 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1093/2206] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:20:17 2022] DELETE /identity/v3/users/196748d4d1744c2d9fc441ea3a061dd9 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:20:17.381121 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ebb62-8e8d-4b25-8748-eed12df0d917 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:20:17.382753 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ebb62-8e8d-4b25-8748-eed12df0d917 service placement] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:20:17.393396 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1114/2207] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:20:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 16:20:17.548750 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ccb20-50c5-411b-be91-26df9b34ae69 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:20:17.550278 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-976ccb20-50c5-411b-be91-26df9b34ae69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:20:17.613871 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ccb20-50c5-411b-be91-26df9b34ae69 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:20:17.618342 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8afdf61-4a31-44a0-8eec-56e7df07c7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:20:17.619192 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8afdf61-4a31-44a0-8eec-56e7df07c7aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:20:17.619986 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8afdf61-4a31-44a0-8eec-56e7df07c7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:20:17.620274 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8afdf61-4a31-44a0-8eec-56e7df07c7aa admin admin] PATH_INFO: `/v3/projects/573ee03c74e3443d8a8d5e5568507d37` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:20:17.621925 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8afdf61-4a31-44a0-8eec-56e7df07c7aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=573ee03c74e3443d8a8d5e5568507d37)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:20:17.623848 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8afdf61-4a31-44a0-8eec-56e7df07c7aa admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:20:17.668169 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d8afdf61-4a31-44a0-8eec-56e7df07c7aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '573ee03c74e3443d8a8d5e5568507d37', 'request_id': 'req-d8afdf61-4a31-44a0-8eec-56e7df07c7aa'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:20:17.674289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d8afdf61-4a31-44a0-8eec-56e7df07c7aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '573ee03c74e3443d8a8d5e5568507d37', 'request_id': 'req-d8afdf61-4a31-44a0-8eec-56e7df07c7aa'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:20:17.682443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d8afdf61-4a31-44a0-8eec-56e7df07c7aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '573ee03c74e3443d8a8d5e5568507d37', 'request_id': 'req-d8afdf61-4a31-44a0-8eec-56e7df07c7aa'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:20:17.687628 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:20:17.695100 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d8afdf61-4a31-44a0-8eec-56e7df07c7aa admin admin] The token cache is being invalidate because project 573ee03c74e3443d8a8d5e5568507d37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:20:17.695433 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d8afdf61-4a31-44a0-8eec-56e7df07c7aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:20:17.696067 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d8afdf61-4a31-44a0-8eec-56e7df07c7aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:20:17.698081 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1094/2208] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:20:17 2022] DELETE /identity/v3/projects/573ee03c74e3443d8a8d5e5568507d37 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:20:17.704635 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e016e78-88d1-4b04-8b7a-1166a5bc76d9 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:20:17.704928 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e016e78-88d1-4b04-8b7a-1166a5bc76d9 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:20:17.705206 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e016e78-88d1-4b04-8b7a-1166a5bc76d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:20:17.723980 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-8e016e78-88d1-4b04-8b7a-1166a5bc76d9 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:20:17.725168 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-8e016e78-88d1-4b04-8b7a-1166a5bc76d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:20:17.782062 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1115/2209] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:20:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:20:17.789558 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0e1d4-3c1e-4f72-8206-60083da3db40 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:20:17.789853 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0e1d4-3c1e-4f72-8206-60083da3db40 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:20:17.790166 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0e1d4-3c1e-4f72-8206-60083da3db40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:20:17.810583 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-05a0e1d4-3c1e-4f72-8206-60083da3db40 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:20:17.811763 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-05a0e1d4-3c1e-4f72-8206-60083da3db40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:20:17.819815 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1095/2210] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:20:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:20:17.823974 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e016e78-88d1-4b04-8b7a-1166a5bc76d9 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:20:17.834176 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e016e78-88d1-4b04-8b7a-1166a5bc76d9 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:20:17.838711 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52013b8-85fa-4288-a981-f9a24fdb226d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:20:17.839461 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52013b8-85fa-4288-a981-f9a24fdb226d admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:20:17.839743 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52013b8-85fa-4288-a981-f9a24fdb226d admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:20:17.840014 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52013b8-85fa-4288-a981-f9a24fdb226d admin admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:20:17.840732 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52013b8-85fa-4288-a981-f9a24fdb226d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:20:17.841037 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52013b8-85fa-4288-a981-f9a24fdb226d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:20:17.842610 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52013b8-85fa-4288-a981-f9a24fdb226d admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:20:17.853739 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1116/2211] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:20:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:20:17.857864 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a0e1d4-3c1e-4f72-8206-60083da3db40 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:20:17.866940 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a0e1d4-3c1e-4f72-8206-60083da3db40 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:20:17.872164 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af623020-11ed-4785-ac66-f989d39f978c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:20:17.872803 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af623020-11ed-4785-ac66-f989d39f978c admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:20:17.873091 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af623020-11ed-4785-ac66-f989d39f978c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:20:17.873373 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af623020-11ed-4785-ac66-f989d39f978c admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:20:17.874059 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af623020-11ed-4785-ac66-f989d39f978c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:20:17.876013 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af623020-11ed-4785-ac66-f989d39f978c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:20:17.903346 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1096/2212] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:20:17 2022] POST /identity/v3/projects => generated 399 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:20:17.905448 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52013b8-85fa-4288-a981-f9a24fdb226d admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:20:17.915171 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52013b8-85fa-4288-a981-f9a24fdb226d admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:20:17.955430 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:20:17.970315 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68e77ee-8ac6-4c4a-8a47-47702d355119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:20:17.970970 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e77ee-8ac6-4c4a-8a47-47702d355119 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:20:17.971252 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e77ee-8ac6-4c4a-8a47-47702d355119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:20:17.971560 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e77ee-8ac6-4c4a-8a47-47702d355119 admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:20:17.972248 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68e77ee-8ac6-4c4a-8a47-47702d355119 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:20:17.974142 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68e77ee-8ac6-4c4a-8a47-47702d355119 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:20:17.993378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1117/2213] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:20:17 2022] POST /identity/v3/users => generated 300 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:20:17.997992 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af623020-11ed-4785-ac66-f989d39f978c admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:20:18.008054 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af623020-11ed-4785-ac66-f989d39f978c admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:20:18.012667 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c795a9-661e-45f1-bedd-79a23477fea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:20:18.013314 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c795a9-661e-45f1-bedd-79a23477fea5 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:20:18.013600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c795a9-661e-45f1-bedd-79a23477fea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:20:18.013873 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c795a9-661e-45f1-bedd-79a23477fea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:20:18.014596 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c795a9-661e-45f1-bedd-79a23477fea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:20:18.016221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c795a9-661e-45f1-bedd-79a23477fea5 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:20:18.026514 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1097/2214] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:20:18.030171 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68e77ee-8ac6-4c4a-8a47-47702d355119 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:20:18.039892 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68e77ee-8ac6-4c4a-8a47-47702d355119 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:20:18.045005 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f73e4-523a-4a32-8dc1-61462b02da12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:20:18.045653 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f73e4-523a-4a32-8dc1-61462b02da12 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:20:18.045941 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f73e4-523a-4a32-8dc1-61462b02da12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:20:18.046224 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f73e4-523a-4a32-8dc1-61462b02da12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:20:18.046943 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f73e4-523a-4a32-8dc1-61462b02da12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:20:18.048560 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f73e4-523a-4a32-8dc1-61462b02da12 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:20:18.058187 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1118/2215] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:20:18.062666 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c795a9-661e-45f1-bedd-79a23477fea5 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:20:18.072353 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c795a9-661e-45f1-bedd-79a23477fea5 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:20:18.076939 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d091e5e-41a1-4d2b-8973-8b6677af6f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:20:18.077801 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d091e5e-41a1-4d2b-8973-8b6677af6f91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:20:18.078084 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d091e5e-41a1-4d2b-8973-8b6677af6f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:20:18.078360 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d091e5e-41a1-4d2b-8973-8b6677af6f91 admin admin] PATH_INFO: `/v3/projects/c24661def0504e89be4e386ad9cc1550/users/a0739f3753a04d4fbf661ce044a776ab/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:20:18.093911 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d091e5e-41a1-4d2b-8973-8b6677af6f91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c24661def0504e89be4e386ad9cc1550, user_id=a0739f3753a04d4fbf661ce044a776ab, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:20:18.096823 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d091e5e-41a1-4d2b-8973-8b6677af6f91 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:20:18.109562 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1098/2216] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:20:18 2022] PUT /identity/v3/projects/c24661def0504e89be4e386ad9cc1550/users/a0739f3753a04d4fbf661ce044a776ab/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:20:18.119963 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee297438-d529-4d1b-984f-c1cc43178fa1 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:20:18.120256 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee297438-d529-4d1b-984f-c1cc43178fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:20:18.120537 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee297438-d529-4d1b-984f-c1cc43178fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:20:18.146950 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-ee297438-d529-4d1b-984f-c1cc43178fa1 None None] MFA Rules not processed for user `a0739f3753a04d4fbf661ce044a776ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:20:18.148199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-ee297438-d529-4d1b-984f-c1cc43178fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:20:18.211193 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1119/2217] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:20:18 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:20:18.222273 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d091e5e-41a1-4d2b-8973-8b6677af6f91 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:20:18.223977 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5d091e5e-41a1-4d2b-8973-8b6677af6f91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:20:18.279167 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d091e5e-41a1-4d2b-8973-8b6677af6f91 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:20:18.282777 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1d1bff-73ee-422c-a7f8-c1ff606a3f73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:20:18.283444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d1bff-73ee-422c-a7f8-c1ff606a3f73 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:20:18.283731 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d1bff-73ee-422c-a7f8-c1ff606a3f73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:20:18.284003 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d1bff-73ee-422c-a7f8-c1ff606a3f73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:20:18.292169 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1d1bff-73ee-422c-a7f8-c1ff606a3f73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:20:18.294135 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1d1bff-73ee-422c-a7f8-c1ff606a3f73 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:20:18.302839 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1099/2218] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:20:18 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:20:28.208296 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=46,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:20:28.298878 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=55,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:20:33.465166 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523a78f-7373-4d69-9b8b-1d8b1eed7ab7 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:20:33.465687 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523a78f-7373-4d69-9b8b-1d8b1eed7ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:20:33.465861 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523a78f-7373-4d69-9b8b-1d8b1eed7ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:20:33.484404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-a523a78f-7373-4d69-9b8b-1d8b1eed7ab7 None None] MFA Rules not processed for user `12a595db7f394c7e86b6a9e0c9f625f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:20:33.485644 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-a523a78f-7373-4d69-9b8b-1d8b1eed7ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:20:33.543911 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1120/2219] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:20:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:20:35.632187 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1d1bff-73ee-422c-a7f8-c1ff606a3f73 service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:20:35.703268 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1d1bff-73ee-422c-a7f8-c1ff606a3f73 service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:20:35.708643 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd83ba-f2e8-4dcf-8ec9-ff5b7f5b34b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:20:35.709523 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd83ba-f2e8-4dcf-8ec9-ff5b7f5b34b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:20:35.709806 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd83ba-f2e8-4dcf-8ec9-ff5b7f5b34b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:20:35.710081 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd83ba-f2e8-4dcf-8ec9-ff5b7f5b34b9 admin admin] PATH_INFO: `/v3/users/a0739f3753a04d4fbf661ce044a776ab` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:20:35.711261 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fd83ba-f2e8-4dcf-8ec9-ff5b7f5b34b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0739f3753a04d4fbf661ce044a776ab)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:20:35.713231 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fd83ba-f2e8-4dcf-8ec9-ff5b7f5b34b9 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:20:35.761017 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a6fd83ba-f2e8-4dcf-8ec9-ff5b7f5b34b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0739f3753a04d4fbf661ce044a776ab'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:20:35.774120 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a6fd83ba-f2e8-4dcf-8ec9-ff5b7f5b34b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0739f3753a04d4fbf661ce044a776ab'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:20:35.789690 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a6fd83ba-f2e8-4dcf-8ec9-ff5b7f5b34b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0739f3753a04d4fbf661ce044a776ab'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:20:35.790420 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a6fd83ba-f2e8-4dcf-8ec9-ff5b7f5b34b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0739f3753a04d4fbf661ce044a776ab'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:20:35.797594 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a6fd83ba-f2e8-4dcf-8ec9-ff5b7f5b34b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0739f3753a04d4fbf661ce044a776ab'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:20:35.807197 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1100/2220] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:20:35 2022] DELETE /identity/v3/users/a0739f3753a04d4fbf661ce044a776ab => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:20:35.808943 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c3df6-c7f3-4193-90bb-c40b13b9a8e2 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:20:35.809416 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c3df6-c7f3-4193-90bb-c40b13b9a8e2 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:20:35.809525 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c3df6-c7f3-4193-90bb-c40b13b9a8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:20:35.827459 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-984c3df6-c7f3-4193-90bb-c40b13b9a8e2 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:20:35.828663 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-984c3df6-c7f3-4193-90bb-c40b13b9a8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:20:35.888368 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1121/2221] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:20:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:20:35.898956 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd83ba-f2e8-4dcf-8ec9-ff5b7f5b34b9 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:20:35.900404 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a6fd83ba-f2e8-4dcf-8ec9-ff5b7f5b34b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:20:35.965236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd83ba-f2e8-4dcf-8ec9-ff5b7f5b34b9 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:20:35.969025 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041992b-68c7-4490-b51b-cd247c88b328 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:20:35.969704 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041992b-68c7-4490-b51b-cd247c88b328 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:20:35.970127 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041992b-68c7-4490-b51b-cd247c88b328 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:20:35.970940 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041992b-68c7-4490-b51b-cd247c88b328 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:20:35.980399 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f041992b-68c7-4490-b51b-cd247c88b328 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:20:35.981901 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f041992b-68c7-4490-b51b-cd247c88b328 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:20:35.992767 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1101/2222] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:20:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:20:36.294488 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c3df6-c7f3-4193-90bb-c40b13b9a8e2 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:20:36.296408 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-984c3df6-c7f3-4193-90bb-c40b13b9a8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:20:36.322352 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c3df6-c7f3-4193-90bb-c40b13b9a8e2 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:20:36.332691 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f5147-96d5-4714-8068-5090a98f2c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:20:36.333602 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f5147-96d5-4714-8068-5090a98f2c68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:20:36.333881 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f5147-96d5-4714-8068-5090a98f2c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:20:36.340377 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f5147-96d5-4714-8068-5090a98f2c68 admin admin] PATH_INFO: `/v3/projects/c24661def0504e89be4e386ad9cc1550` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:20:36.341660 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f5147-96d5-4714-8068-5090a98f2c68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c24661def0504e89be4e386ad9cc1550)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:20:36.343605 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f5147-96d5-4714-8068-5090a98f2c68 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:20:36.420636 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-b09f5147-96d5-4714-8068-5090a98f2c68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c24661def0504e89be4e386ad9cc1550', 'request_id': 'req-b09f5147-96d5-4714-8068-5090a98f2c68'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:20:36.429027 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-b09f5147-96d5-4714-8068-5090a98f2c68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c24661def0504e89be4e386ad9cc1550', 'request_id': 'req-b09f5147-96d5-4714-8068-5090a98f2c68'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:20:36.444646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-b09f5147-96d5-4714-8068-5090a98f2c68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c24661def0504e89be4e386ad9cc1550', 'request_id': 'req-b09f5147-96d5-4714-8068-5090a98f2c68'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:20:36.460624 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-b09f5147-96d5-4714-8068-5090a98f2c68 admin admin] The token cache is being invalidate because project c24661def0504e89be4e386ad9cc1550 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:20:36.460903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-b09f5147-96d5-4714-8068-5090a98f2c68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:20:36.461634 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-b09f5147-96d5-4714-8068-5090a98f2c68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:20:36.483421 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1122/2223] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:20:36 2022] DELETE /identity/v3/projects/c24661def0504e89be4e386ad9cc1550 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:20:43.397799 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041992b-68c7-4490-b51b-cd247c88b328 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:20:43.408341 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041992b-68c7-4490-b51b-cd247c88b328 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:20:43.437062 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:20:43.472510 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d925a-c62c-4601-b01b-e6875b9fbc1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:20:43.473083 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d925a-c62c-4601-b01b-e6875b9fbc1f service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:20:43.473368 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d925a-c62c-4601-b01b-e6875b9fbc1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:20:43.473648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d925a-c62c-4601-b01b-e6875b9fbc1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:20:43.475466 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-759d925a-c62c-4601-b01b-e6875b9fbc1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:20:43.542713 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759d925a-c62c-4601-b01b-e6875b9fbc1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:20:43.544192 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759d925a-c62c-4601-b01b-e6875b9fbc1f service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:20:43.555319 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1102/2224] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:20:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:20:46.455363 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:20:48.240270 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f5147-96d5-4714-8068-5090a98f2c68 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:20:48.241751 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-b09f5147-96d5-4714-8068-5090a98f2c68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:20:48.253897 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f5147-96d5-4714-8068-5090a98f2c68 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:20:48.257477 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad727d1-5086-446e-8b7e-d7f8f03ee569 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:20:48.258116 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad727d1-5086-446e-8b7e-d7f8f03ee569 service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:20:48.258404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad727d1-5086-446e-8b7e-d7f8f03ee569 service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:20:48.258687 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad727d1-5086-446e-8b7e-d7f8f03ee569 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:20:48.260425 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-4ad727d1-5086-446e-8b7e-d7f8f03ee569 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:20:48.274282 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad727d1-5086-446e-8b7e-d7f8f03ee569 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:20:48.275748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad727d1-5086-446e-8b7e-d7f8f03ee569 service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:20:48.287100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1123/2225] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:20:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:20:50.860212 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d925a-c62c-4601-b01b-e6875b9fbc1f service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:20:50.862821 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-759d925a-c62c-4601-b01b-e6875b9fbc1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:20:50.879905 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d925a-c62c-4601-b01b-e6875b9fbc1f service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:20:50.890813 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c58e7-ee1e-4a80-8a0b-fb73eff7c3cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:20:50.891765 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c58e7-ee1e-4a80-8a0b-fb73eff7c3cf service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:20:50.892024 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c58e7-ee1e-4a80-8a0b-fb73eff7c3cf service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:20:50.892323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c58e7-ee1e-4a80-8a0b-fb73eff7c3cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:20:50.894374 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-726c58e7-ee1e-4a80-8a0b-fb73eff7c3cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:20:50.966933 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c58e7-ee1e-4a80-8a0b-fb73eff7c3cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:20:50.968648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c58e7-ee1e-4a80-8a0b-fb73eff7c3cf service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:20:50.978665 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1103/2226] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:20:50 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:20:52.577539 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c2a5a-71ec-48ec-a5e7-4ff88f807423 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:20:52.578038 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c2a5a-71ec-48ec-a5e7-4ff88f807423 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:20:52.578152 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c2a5a-71ec-48ec-a5e7-4ff88f807423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:20:52.597531 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-6e6c2a5a-71ec-48ec-a5e7-4ff88f807423 None None] MFA Rules not processed for user `ef7c1e449b864b50b3c71c39f19968a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:20:52.599502 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-6e6c2a5a-71ec-48ec-a5e7-4ff88f807423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:20:52.657985 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1124/2227] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:20:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:20:52.668977 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c58e7-ee1e-4a80-8a0b-fb73eff7c3cf service nova] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:20:52.679907 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c58e7-ee1e-4a80-8a0b-fb73eff7c3cf service nova] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:20:52.685361 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a218e860-bbcb-4c67-a338-5c146792195a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:20:52.686027 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a218e860-bbcb-4c67-a338-5c146792195a service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:20:52.686309 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a218e860-bbcb-4c67-a338-5c146792195a service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:20:52.686585 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a218e860-bbcb-4c67-a338-5c146792195a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:20:52.688541 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a218e860-bbcb-4c67-a338-5c146792195a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:20:52.699321 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a218e860-bbcb-4c67-a338-5c146792195a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:20:52.700920 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a218e860-bbcb-4c67-a338-5c146792195a service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:20:52.710885 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1104/2228] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:20:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:20:53.002000 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c2a5a-71ec-48ec-a5e7-4ff88f807423 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:20:53.004037 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-6e6c2a5a-71ec-48ec-a5e7-4ff88f807423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:20:53.015211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c2a5a-71ec-48ec-a5e7-4ff88f807423 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:20:53.019353 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a1f35-3ef3-410b-a12f-9837ea051bfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:20:53.019988 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a1f35-3ef3-410b-a12f-9837ea051bfa service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:20:53.020269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a1f35-3ef3-410b-a12f-9837ea051bfa service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:20:53.020544 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a1f35-3ef3-410b-a12f-9837ea051bfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:20:53.022213 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-d38a1f35-3ef3-410b-a12f-9837ea051bfa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:20:53.031484 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a1f35-3ef3-410b-a12f-9837ea051bfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:20:53.032882 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a1f35-3ef3-410b-a12f-9837ea051bfa service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:20:53.040568 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1125/2229] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:20:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:20:53.344435 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a218e860-bbcb-4c67-a338-5c146792195a service nova] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:20:53.345903 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a218e860-bbcb-4c67-a338-5c146792195a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:20:53.402333 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:20:53.422310 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a218e860-bbcb-4c67-a338-5c146792195a service nova] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:20:53.427875 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d37bdd-2ae5-4ea5-9881-a2d45f9a05a7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be901786f8934d559e0bac5d622f97cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:20:53.428515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d37bdd-2ae5-4ea5-9881-a2d45f9a05a7 service placement] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:20:53.428798 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d37bdd-2ae5-4ea5-9881-a2d45f9a05a7 service placement] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:20:53.429074 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d37bdd-2ae5-4ea5-9881-a2d45f9a05a7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:20:53.430836 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-86d37bdd-2ae5-4ea5-9881-a2d45f9a05a7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:20:53.441806 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d37bdd-2ae5-4ea5-9881-a2d45f9a05a7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:20:53.443394 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d37bdd-2ae5-4ea5-9881-a2d45f9a05a7 service placement] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:20:53.452858 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1105/2230] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:20:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:21:02.048782 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a1f35-3ef3-410b-a12f-9837ea051bfa service neutron] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:21:02.063199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a1f35-3ef3-410b-a12f-9837ea051bfa service neutron] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:21:02.068261 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53883c1d-c539-4006-81f2-162d215741cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:21:02.068919 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53883c1d-c539-4006-81f2-162d215741cd service nova] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:02.069191 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53883c1d-c539-4006-81f2-162d215741cd service nova] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:02.069470 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53883c1d-c539-4006-81f2-162d215741cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:02.071249 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-53883c1d-c539-4006-81f2-162d215741cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:21:02.132229 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53883c1d-c539-4006-81f2-162d215741cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:21:02.133760 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53883c1d-c539-4006-81f2-162d215741cd service nova] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:21:02.143322 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1126/2231] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:21:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:21:03.448776 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=10 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:21:12.138877 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=65 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:21:15.428643 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d37bdd-2ae5-4ea5-9881-a2d45f9a05a7 service placement] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:21:15.437984 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d37bdd-2ae5-4ea5-9881-a2d45f9a05a7 service placement] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:21:15.441583 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ddd5a-3aa0-41e7-bed9-f4142160a600 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:21:15.442330 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ddd5a-3aa0-41e7-bed9-f4142160a600 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:15.442614 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ddd5a-3aa0-41e7-bed9-f4142160a600 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:15.442973 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ddd5a-3aa0-41e7-bed9-f4142160a600 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:15.444804 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-800ddd5a-3aa0-41e7-bed9-f4142160a600 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:21:15.454017 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ddd5a-3aa0-41e7-bed9-f4142160a600 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:21:15.455468 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ddd5a-3aa0-41e7-bed9-f4142160a600 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:21:15.464358 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1106/2232] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:21:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:21:19.464649 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70696d-dd58-47d0-802d-10ccb3ade0ad None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:19.465120 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70696d-dd58-47d0-802d-10ccb3ade0ad None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:19.465210 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70696d-dd58-47d0-802d-10ccb3ade0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:19.483500 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-af70696d-dd58-47d0-802d-10ccb3ade0ad None None] MFA Rules not processed for user `875140cf51d9410da45f8c3805b26204`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:21:19.485402 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-af70696d-dd58-47d0-802d-10ccb3ade0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:21:19.541680 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1127/2233] 10.222.0.41 () {58 vars in 1009 bytes} [Fri Jun 3 16:21:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:21:19.587937 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ddd5a-3aa0-41e7-bed9-f4142160a600 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:21:19.589366 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-800ddd5a-3aa0-41e7-bed9-f4142160a600 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:21:19.658836 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ddd5a-3aa0-41e7-bed9-f4142160a600 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:21:19.664465 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109fefa-9b58-427b-8c89-569763216ee2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5f513b781a4a73b443cded13359eda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:21:19.665109 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109fefa-9b58-427b-8c89-569763216ee2 service swift] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:19.665408 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109fefa-9b58-427b-8c89-569763216ee2 service swift] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:19.665683 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109fefa-9b58-427b-8c89-569763216ee2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:19.674038 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109fefa-9b58-427b-8c89-569763216ee2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:21:19.676066 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109fefa-9b58-427b-8c89-569763216ee2 service swift] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:21:19.684255 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1107/2234] 10.222.0.41 () {60 vars in 1384 bytes} [Fri Jun 3 16:21:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 16:21:24.986124 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af70696d-dd58-47d0-802d-10ccb3ade0ad None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:21:25.003331 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af70696d-dd58-47d0-802d-10ccb3ade0ad None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:21:25.012222 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d89920-93aa-412f-994c-a88c66a73bd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:21:25.013076 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d89920-93aa-412f-994c-a88c66a73bd8 service nova] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:25.013518 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d89920-93aa-412f-994c-a88c66a73bd8 service nova] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:25.013982 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d89920-93aa-412f-994c-a88c66a73bd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:25.015993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-28d89920-93aa-412f-994c-a88c66a73bd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:21:25.029055 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d89920-93aa-412f-994c-a88c66a73bd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:21:25.030650 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d89920-93aa-412f-994c-a88c66a73bd8 service nova] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:21:25.044489 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1128/2235] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:21:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:21:29.680829 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=38 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:21:35.038558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=31 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:21:36.844634 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109fefa-9b58-427b-8c89-569763216ee2 service swift] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:21:36.853671 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109fefa-9b58-427b-8c89-569763216ee2 service swift] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:21:36.857239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c221be6d-b6c8-4266-8a05-81628690ab98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:21:36.857874 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221be6d-b6c8-4266-8a05-81628690ab98 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:36.858158 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221be6d-b6c8-4266-8a05-81628690ab98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:36.858435 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221be6d-b6c8-4266-8a05-81628690ab98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:36.860233 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c221be6d-b6c8-4266-8a05-81628690ab98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:21:36.869899 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c221be6d-b6c8-4266-8a05-81628690ab98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:21:36.871379 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c221be6d-b6c8-4266-8a05-81628690ab98 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:21:36.879034 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1108/2236] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:21:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:21:36.980277 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee1b9d-2e3f-4a4d-8313-7786d7340e73 None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:36.980751 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee1b9d-2e3f-4a4d-8313-7786d7340e73 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:36.980858 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee1b9d-2e3f-4a4d-8313-7786d7340e73 None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:36.982299 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1129/2237] 10.222.0.41 () {54 vars in 921 bytes} [Fri Jun 3 16:21:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:21:36.990532 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4db998-0466-47dc-b0dd-28b40b43eae2 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:36.990826 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4db998-0466-47dc-b0dd-28b40b43eae2 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:36.991106 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4db998-0466-47dc-b0dd-28b40b43eae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:37.029878 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-ae4db998-0466-47dc-b0dd-28b40b43eae2 None None] MFA Rules not processed for user `2fd08ea39ee94b92ac6763608e69f158`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:21:37.031088 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ae4db998-0466-47dc-b0dd-28b40b43eae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:21:37.042395 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1109/2238] 10.222.0.41 () {58 vars in 1018 bytes} [Fri Jun 3 16:21:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:21:37.052891 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee1b9d-2e3f-4a4d-8313-7786d7340e73 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:21:37.054630 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-5aee1b9d-2e3f-4a4d-8313-7786d7340e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:21:37.147618 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee1b9d-2e3f-4a4d-8313-7786d7340e73 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:21:37.151095 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c8c77-71b4-41b5-a478-3b120fb21bcb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbaf66d6e1f54047917cc615546bbd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:21:37.151759 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c8c77-71b4-41b5-a478-3b120fb21bcb service glance] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:37.152041 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c8c77-71b4-41b5-a478-3b120fb21bcb service glance] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:37.152314 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c8c77-71b4-41b5-a478-3b120fb21bcb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:37.162650 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c8c77-71b4-41b5-a478-3b120fb21bcb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:21:37.164101 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c8c77-71b4-41b5-a478-3b120fb21bcb service glance] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:21:37.174515 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1130/2239] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:21:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:21:37.241267 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab93e6-6921-4ac2-92ea-7f99e0afc658 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:37.241558 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab93e6-6921-4ac2-92ea-7f99e0afc658 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:37.241835 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab93e6-6921-4ac2-92ea-7f99e0afc658 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:37.243201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1110/2240] 10.222.0.41 () {54 vars in 921 bytes} [Fri Jun 3 16:21:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:21:37.247937 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584bbd8-dddc-46f3-b874-2a4299448b0a None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:37.248333 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584bbd8-dddc-46f3-b874-2a4299448b0a None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:37.248674 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584bbd8-dddc-46f3-b874-2a4299448b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:37.266641 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-8584bbd8-dddc-46f3-b874-2a4299448b0a None None] MFA Rules not processed for user `aeeaad0680074179b3d29abcf0d51515`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:21:37.267861 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-8584bbd8-dddc-46f3-b874-2a4299448b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:21:37.274817 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1131/2241] 10.222.0.41 () {58 vars in 1018 bytes} [Fri Jun 3 16:21:37 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:21:37.286802 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab93e6-6921-4ac2-92ea-7f99e0afc658 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:21:37.297472 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab93e6-6921-4ac2-92ea-7f99e0afc658 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:21:37.301009 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5056111-0166-4f81-9f45-1455f52997ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:21:37.301640 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5056111-0166-4f81-9f45-1455f52997ec service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:37.302483 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5056111-0166-4f81-9f45-1455f52997ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:37.302767 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5056111-0166-4f81-9f45-1455f52997ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:37.309697 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5056111-0166-4f81-9f45-1455f52997ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:21:37.311121 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5056111-0166-4f81-9f45-1455f52997ec service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:21:37.318886 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1111/2242] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:21:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:21:37.399002 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e12ec4-6404-408f-b7e9-43a0787185ee None None] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:37.399322 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e12ec4-6404-408f-b7e9-43a0787185ee None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:37.399616 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e12ec4-6404-408f-b7e9-43a0787185ee None None] PATH_INFO: `/` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:37.400984 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1132/2243] 10.222.0.41 () {54 vars in 921 bytes} [Fri Jun 3 16:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 16:21:37.405763 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cca7dd-df1b-438d-9de1-dbcf5befffe7 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:37.406063 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cca7dd-df1b-438d-9de1-dbcf5befffe7 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:37.406349 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cca7dd-df1b-438d-9de1-dbcf5befffe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:37.425257 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-25cca7dd-df1b-438d-9de1-dbcf5befffe7 None None] MFA Rules not processed for user `2fd08ea39ee94b92ac6763608e69f158`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:21:37.426493 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-25cca7dd-df1b-438d-9de1-dbcf5befffe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:21:37.436480 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1112/2244] 10.222.0.41 () {58 vars in 1018 bytes} [Fri Jun 3 16:21:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:21:37.446666 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e12ec4-6404-408f-b7e9-43a0787185ee None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:21:37.457659 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e12ec4-6404-408f-b7e9-43a0787185ee None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:21:37.461267 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50232f4d-62ab-4cc0-b664-28f54162128f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:21:37.461895 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50232f4d-62ab-4cc0-b664-28f54162128f service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:37.462179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50232f4d-62ab-4cc0-b664-28f54162128f service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:37.462454 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50232f4d-62ab-4cc0-b664-28f54162128f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:37.472292 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50232f4d-62ab-4cc0-b664-28f54162128f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:21:37.474184 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50232f4d-62ab-4cc0-b664-28f54162128f service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:21:37.484695 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1133/2245] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:21:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 16:21:38.712615 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cca7dd-df1b-438d-9de1-dbcf5befffe7 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:21:38.724538 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cca7dd-df1b-438d-9de1-dbcf5befffe7 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:21:38.730188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18aed3-0cea-422a-9104-6af399253516 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:21:38.730884 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18aed3-0cea-422a-9104-6af399253516 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:38.731194 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18aed3-0cea-422a-9104-6af399253516 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:38.731533 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18aed3-0cea-422a-9104-6af399253516 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:38.733360 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5c18aed3-0cea-422a-9104-6af399253516 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:21:38.746131 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c18aed3-0cea-422a-9104-6af399253516 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:21:38.747792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c18aed3-0cea-422a-9104-6af399253516 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:21:38.758262 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1113/2246] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:21:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 16:21:44.168461 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ba43c-a832-4571-9a48-9894c7a65fa7 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:44.168935 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ba43c-a832-4571-9a48-9894c7a65fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:44.169031 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ba43c-a832-4571-9a48-9894c7a65fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:44.186321 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-d84ba43c-a832-4571-9a48-9894c7a65fa7 None None] MFA Rules not processed for user `875140cf51d9410da45f8c3805b26204`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:21:44.187542 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-d84ba43c-a832-4571-9a48-9894c7a65fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:21:44.195194 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1134/2247] 10.222.0.41 () {58 vars in 1009 bytes} [Fri Jun 3 16:21:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:21:44.207046 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18aed3-0cea-422a-9104-6af399253516 service nova] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:21:44.219744 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18aed3-0cea-422a-9104-6af399253516 service nova] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:21:44.224714 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b7670-2e2b-4ba5-bce5-9e59b852b211 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5f513b781a4a73b443cded13359eda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:21:44.225348 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b7670-2e2b-4ba5-bce5-9e59b852b211 service swift] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:44.225629 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b7670-2e2b-4ba5-bce5-9e59b852b211 service swift] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:44.225898 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b7670-2e2b-4ba5-bce5-9e59b852b211 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:44.234186 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8b7670-2e2b-4ba5-bce5-9e59b852b211 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:21:44.235779 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8b7670-2e2b-4ba5-bce5-9e59b852b211 service swift] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:21:44.244128 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1114/2248] 10.222.0.41 () {60 vars in 1384 bytes} [Fri Jun 3 16:21:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 16:21:53.250406 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ba43c-a832-4571-9a48-9894c7a65fa7 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:21:53.252660 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-d84ba43c-a832-4571-9a48-9894c7a65fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:21:53.332899 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ba43c-a832-4571-9a48-9894c7a65fa7 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:21:53.337967 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd4f26-1d5c-4bb2-acbc-d6a14dd8441a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:21:53.338840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd4f26-1d5c-4bb2-acbc-d6a14dd8441a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:53.339127 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd4f26-1d5c-4bb2-acbc-d6a14dd8441a admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:53.339437 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd4f26-1d5c-4bb2-acbc-d6a14dd8441a admin admin] PATH_INFO: `/v3/users/12a595db7f394c7e86b6a9e0c9f625f8` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:53.340646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fd4f26-1d5c-4bb2-acbc-d6a14dd8441a admin admin] RBAC: Authorizing `identity:delete_user(user_id=12a595db7f394c7e86b6a9e0c9f625f8)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:21:53.342574 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fd4f26-1d5c-4bb2-acbc-d6a14dd8441a admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:21:53.408802 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-29fd4f26-1d5c-4bb2-acbc-d6a14dd8441a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12a595db7f394c7e86b6a9e0c9f625f8'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:21:53.433038 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-29fd4f26-1d5c-4bb2-acbc-d6a14dd8441a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12a595db7f394c7e86b6a9e0c9f625f8'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:21:53.439118 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:21:53.452106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-29fd4f26-1d5c-4bb2-acbc-d6a14dd8441a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12a595db7f394c7e86b6a9e0c9f625f8'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:21:53.463054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-29fd4f26-1d5c-4bb2-acbc-d6a14dd8441a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12a595db7f394c7e86b6a9e0c9f625f8'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:21:53.463840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-29fd4f26-1d5c-4bb2-acbc-d6a14dd8441a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12a595db7f394c7e86b6a9e0c9f625f8'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:21:53.471525 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1135/2249] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:21:53 2022] DELETE /identity/v3/users/12a595db7f394c7e86b6a9e0c9f625f8 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:21:53.674619 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b7670-2e2b-4ba5-bce5-9e59b852b211 service swift] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:21:53.690757 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b7670-2e2b-4ba5-bce5-9e59b852b211 service swift] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:21:53.772323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6423050-2878-43d2-97be-bbd1f8c922e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:21:53.773184 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6423050-2878-43d2-97be-bbd1f8c922e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:53.773469 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6423050-2878-43d2-97be-bbd1f8c922e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:53.773745 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6423050-2878-43d2-97be-bbd1f8c922e8 admin admin] PATH_INFO: `/v3/projects/1b5c201aae5e438c81d8976289ffb4cb` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:53.774958 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6423050-2878-43d2-97be-bbd1f8c922e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b5c201aae5e438c81d8976289ffb4cb)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:21:53.776889 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6423050-2878-43d2-97be-bbd1f8c922e8 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:21:53.826207 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a6423050-2878-43d2-97be-bbd1f8c922e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b5c201aae5e438c81d8976289ffb4cb', 'request_id': 'req-a6423050-2878-43d2-97be-bbd1f8c922e8'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:21:53.833518 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a6423050-2878-43d2-97be-bbd1f8c922e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b5c201aae5e438c81d8976289ffb4cb', 'request_id': 'req-a6423050-2878-43d2-97be-bbd1f8c922e8'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:21:53.842538 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a6423050-2878-43d2-97be-bbd1f8c922e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b5c201aae5e438c81d8976289ffb4cb', 'request_id': 'req-a6423050-2878-43d2-97be-bbd1f8c922e8'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:21:53.855366 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a6423050-2878-43d2-97be-bbd1f8c922e8 admin admin] The token cache is being invalidate because project 1b5c201aae5e438c81d8976289ffb4cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:21:53.855660 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a6423050-2878-43d2-97be-bbd1f8c922e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:21:53.856324 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a6423050-2878-43d2-97be-bbd1f8c922e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:21:53.857929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1115/2250] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:21:53 2022] DELETE /identity/v3/projects/1b5c201aae5e438c81d8976289ffb4cb => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:21:53.863039 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b667258-55a7-49c6-979b-c0fa78aab32c None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:53.863368 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b667258-55a7-49c6-979b-c0fa78aab32c None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:53.863648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b667258-55a7-49c6-979b-c0fa78aab32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:53.881870 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-4b667258-55a7-49c6-979b-c0fa78aab32c None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:21:53.883051 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-4b667258-55a7-49c6-979b-c0fa78aab32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:21:53.950564 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1136/2251] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:21:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:21:53.967222 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9167ba6-2b96-4a1c-bc8a-b4a01aa4d674 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:53.969261 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9167ba6-2b96-4a1c-bc8a-b4a01aa4d674 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:53.971606 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9167ba6-2b96-4a1c-bc8a-b4a01aa4d674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:54.021297 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-d9167ba6-2b96-4a1c-bc8a-b4a01aa4d674 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:21:54.022897 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d9167ba6-2b96-4a1c-bc8a-b4a01aa4d674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:21:54.031336 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1116/2252] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:21:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:21:54.037307 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b667258-55a7-49c6-979b-c0fa78aab32c None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:21:54.049332 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b667258-55a7-49c6-979b-c0fa78aab32c None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:21:54.058754 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d1a3b-6ab6-4c6c-963b-56e7e2d638f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:21:54.059702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d1a3b-6ab6-4c6c-963b-56e7e2d638f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:54.060185 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d1a3b-6ab6-4c6c-963b-56e7e2d638f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:54.060559 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d1a3b-6ab6-4c6c-963b-56e7e2d638f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:54.061501 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d1a3b-6ab6-4c6c-963b-56e7e2d638f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:21:54.062167 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d1a3b-6ab6-4c6c-963b-56e7e2d638f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:21:54.064150 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d1a3b-6ab6-4c6c-963b-56e7e2d638f8 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:21:54.084517 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1137/2253] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:21:54.090875 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9167ba6-2b96-4a1c-bc8a-b4a01aa4d674 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:21:54.097493 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:21:54.110311 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9167ba6-2b96-4a1c-bc8a-b4a01aa4d674 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:21:54.119236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ccb3cd-185d-408f-837c-bee790c873bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:21:54.123746 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ccb3cd-185d-408f-837c-bee790c873bf admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:54.123746 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ccb3cd-185d-408f-837c-bee790c873bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:54.123746 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ccb3cd-185d-408f-837c-bee790c873bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:54.123746 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ccb3cd-185d-408f-837c-bee790c873bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:21:54.123746 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ccb3cd-185d-408f-837c-bee790c873bf admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:21:54.157782 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1117/2254] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:21:54 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:21:54.162307 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d1a3b-6ab6-4c6c-963b-56e7e2d638f8 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:21:54.172879 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d1a3b-6ab6-4c6c-963b-56e7e2d638f8 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:21:54.240249 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e93e77-52b7-498b-baad-2a3584faaed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:21:54.240843 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e93e77-52b7-498b-baad-2a3584faaed1 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:54.241134 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e93e77-52b7-498b-baad-2a3584faaed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:54.241415 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e93e77-52b7-498b-baad-2a3584faaed1 admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:54.242155 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e93e77-52b7-498b-baad-2a3584faaed1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:21:54.244086 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e93e77-52b7-498b-baad-2a3584faaed1 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:21:54.267934 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1138/2255] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:21:54 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:21:54.269092 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ccb3cd-185d-408f-837c-bee790c873bf admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:21:54.280543 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ccb3cd-185d-408f-837c-bee790c873bf admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:21:54.287510 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b85d26-101f-43eb-b869-674ba095364f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:21:54.288264 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b85d26-101f-43eb-b869-674ba095364f admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:54.288557 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b85d26-101f-43eb-b869-674ba095364f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:54.288844 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b85d26-101f-43eb-b869-674ba095364f admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:54.289757 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b85d26-101f-43eb-b869-674ba095364f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:21:54.291413 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b85d26-101f-43eb-b869-674ba095364f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:21:54.303753 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1118/2256] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:21:54.307983 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e93e77-52b7-498b-baad-2a3584faaed1 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:21:54.320123 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e93e77-52b7-498b-baad-2a3584faaed1 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:21:54.330637 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bead80-b700-48ee-a8a9-d5b6da4ed65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:21:54.331378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bead80-b700-48ee-a8a9-d5b6da4ed65d admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:54.331641 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bead80-b700-48ee-a8a9-d5b6da4ed65d admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:54.331922 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bead80-b700-48ee-a8a9-d5b6da4ed65d admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:54.332663 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bead80-b700-48ee-a8a9-d5b6da4ed65d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:21:54.334351 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bead80-b700-48ee-a8a9-d5b6da4ed65d admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:21:54.345861 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1139/2257] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:21:54.350504 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b85d26-101f-43eb-b869-674ba095364f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:21:54.360368 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b85d26-101f-43eb-b869-674ba095364f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:21:54.365632 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e75477-39ea-46f5-9bbb-a9299c3db6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:21:54.366530 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e75477-39ea-46f5-9bbb-a9299c3db6b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:54.366814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e75477-39ea-46f5-9bbb-a9299c3db6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:54.367090 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e75477-39ea-46f5-9bbb-a9299c3db6b9 admin admin] PATH_INFO: `/v3/projects/566b367dc7504d3091558cfd6f35bfa9/users/1e415e30c70d441991187b13c229ee18/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:54.381088 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e75477-39ea-46f5-9bbb-a9299c3db6b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=566b367dc7504d3091558cfd6f35bfa9, user_id=1e415e30c70d441991187b13c229ee18, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:21:54.383775 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e75477-39ea-46f5-9bbb-a9299c3db6b9 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:21:54.398667 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1119/2258] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:21:54 2022] PUT /identity/v3/projects/566b367dc7504d3091558cfd6f35bfa9/users/1e415e30c70d441991187b13c229ee18/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:21:54.403808 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdf439-c1da-48c0-975f-3d8954cca066 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:54.404185 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdf439-c1da-48c0-975f-3d8954cca066 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:54.404675 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdf439-c1da-48c0-975f-3d8954cca066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:54.427657 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-a2fdf439-c1da-48c0-975f-3d8954cca066 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:21:54.428896 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-a2fdf439-c1da-48c0-975f-3d8954cca066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:21:54.468983 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:21:54.492475 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1140/2259] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:21:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:21:54.502011 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e75477-39ea-46f5-9bbb-a9299c3db6b9 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:21:54.503748 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-39e75477-39ea-46f5-9bbb-a9299c3db6b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:21:54.561282 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e75477-39ea-46f5-9bbb-a9299c3db6b9 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:21:54.565533 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a2d123-9fc0-4ab1-b308-57c4db7d8cdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:21:54.566294 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2d123-9fc0-4ab1-b308-57c4db7d8cdd service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:54.566590 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2d123-9fc0-4ab1-b308-57c4db7d8cdd service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:54.566882 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2d123-9fc0-4ab1-b308-57c4db7d8cdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:54.580066 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a2d123-9fc0-4ab1-b308-57c4db7d8cdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:21:54.581520 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a2d123-9fc0-4ab1-b308-57c4db7d8cdd service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:21:54.601145 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1120/2260] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:21:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:21:59.355393 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f99fab-bbf2-4d5f-b129-f86c95178a4e None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:59.356028 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f99fab-bbf2-4d5f-b129-f86c95178a4e None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:59.356365 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f99fab-bbf2-4d5f-b129-f86c95178a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:59.386875 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-d8f99fab-bbf2-4d5f-b129-f86c95178a4e None None] MFA Rules not processed for user `1e415e30c70d441991187b13c229ee18`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:21:59.388243 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-d8f99fab-bbf2-4d5f-b129-f86c95178a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:21:59.462897 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1141/2261] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:21:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:21:59.467592 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a2d123-9fc0-4ab1-b308-57c4db7d8cdd service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:21:59.478135 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a2d123-9fc0-4ab1-b308-57c4db7d8cdd service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:21:59.483050 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c43fccc-ba8d-4df5-b37b-20e94d40c0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:21:59.483830 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43fccc-ba8d-4df5-b37b-20e94d40c0c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:59.484177 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43fccc-ba8d-4df5-b37b-20e94d40c0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:59.484518 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43fccc-ba8d-4df5-b37b-20e94d40c0c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:59.485295 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c43fccc-ba8d-4df5-b37b-20e94d40c0c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:21:59.487427 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c43fccc-ba8d-4df5-b37b-20e94d40c0c3 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:21:59.531358 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1121/2262] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:21:59 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:21:59.535720 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f99fab-bbf2-4d5f-b129-f86c95178a4e None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:21:59.550707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f99fab-bbf2-4d5f-b129-f86c95178a4e None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:21:59.623517 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166ad9c-5a88-4952-aea6-f95961ea201c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:21:59.624409 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166ad9c-5a88-4952-aea6-f95961ea201c admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:59.624688 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166ad9c-5a88-4952-aea6-f95961ea201c admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:59.625089 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166ad9c-5a88-4952-aea6-f95961ea201c admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:59.625794 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a166ad9c-5a88-4952-aea6-f95961ea201c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:21:59.629065 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a166ad9c-5a88-4952-aea6-f95961ea201c admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:21:59.657758 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1142/2263] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:21:59 2022] POST /identity/v3/users => generated 299 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:21:59.658943 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c43fccc-ba8d-4df5-b37b-20e94d40c0c3 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:21:59.673707 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c43fccc-ba8d-4df5-b37b-20e94d40c0c3 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:21:59.680146 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c633cf95-5d4e-4340-8f2e-2ee3e8f0d898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:21:59.680822 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633cf95-5d4e-4340-8f2e-2ee3e8f0d898 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:59.681158 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633cf95-5d4e-4340-8f2e-2ee3e8f0d898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:59.681441 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633cf95-5d4e-4340-8f2e-2ee3e8f0d898 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:59.682317 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c633cf95-5d4e-4340-8f2e-2ee3e8f0d898 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:21:59.684012 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c633cf95-5d4e-4340-8f2e-2ee3e8f0d898 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:21:59.696263 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1122/2264] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:21:59.701303 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166ad9c-5a88-4952-aea6-f95961ea201c admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:21:59.712445 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166ad9c-5a88-4952-aea6-f95961ea201c admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:21:59.717299 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f0c2e-8024-4c23-8a90-f2cd898b2707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:21:59.718484 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f0c2e-8024-4c23-8a90-f2cd898b2707 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:59.718841 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f0c2e-8024-4c23-8a90-f2cd898b2707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:59.719329 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f0c2e-8024-4c23-8a90-f2cd898b2707 admin admin] PATH_INFO: `/v3/projects/ecea87a215f24b0abcf9d18797a25521/users/0d93485016824928ab7d2550b290c0c7/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:59.736739 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f0c2e-8024-4c23-8a90-f2cd898b2707 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecea87a215f24b0abcf9d18797a25521, user_id=0d93485016824928ab7d2550b290c0c7, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:21:59.739463 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f0c2e-8024-4c23-8a90-f2cd898b2707 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:21:59.752827 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1143/2265] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:21:59 2022] PUT /identity/v3/projects/ecea87a215f24b0abcf9d18797a25521/users/0d93485016824928ab7d2550b290c0c7/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:21:59.757620 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c633cf95-5d4e-4340-8f2e-2ee3e8f0d898 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:21:59.768077 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c633cf95-5d4e-4340-8f2e-2ee3e8f0d898 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:21:59.828951 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d2a60-6730-473a-92d1-6c4eff131ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:21:59.829621 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d2a60-6730-473a-92d1-6c4eff131ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:59.829904 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d2a60-6730-473a-92d1-6c4eff131ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:59.830181 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d2a60-6730-473a-92d1-6c4eff131ed2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:59.830909 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9d2a60-6730-473a-92d1-6c4eff131ed2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:21:59.832552 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9d2a60-6730-473a-92d1-6c4eff131ed2 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:21:59.843628 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1123/2266] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:21:59.847754 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f0c2e-8024-4c23-8a90-f2cd898b2707 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:21:59.857877 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f0c2e-8024-4c23-8a90-f2cd898b2707 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:21:59.862444 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b78d13-7dad-4a5e-b349-f74782c5c6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:21:59.863323 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b78d13-7dad-4a5e-b349-f74782c5c6e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:21:59.863616 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b78d13-7dad-4a5e-b349-f74782c5c6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:21:59.863889 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b78d13-7dad-4a5e-b349-f74782c5c6e2 admin admin] PATH_INFO: `/v3/projects/ecea87a215f24b0abcf9d18797a25521/users/0d93485016824928ab7d2550b290c0c7/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:21:59.865836 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b78d13-7dad-4a5e-b349-f74782c5c6e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecea87a215f24b0abcf9d18797a25521, user_id=0d93485016824928ab7d2550b290c0c7, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:21:59.868524 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b78d13-7dad-4a5e-b349-f74782c5c6e2 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:21:59.878674 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1144/2267] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:21:59 2022] PUT /identity/v3/projects/ecea87a215f24b0abcf9d18797a25521/users/0d93485016824928ab7d2550b290c0c7/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:22:04.899767 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ffc52-dbca-4a4b-8538-3d93cbe7e9d1 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:22:04.900192 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ffc52-dbca-4a4b-8538-3d93cbe7e9d1 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:22:04.900192 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ffc52-dbca-4a4b-8538-3d93cbe7e9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:22:04.907383 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:22:04.931030 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-000ffc52-dbca-4a4b-8538-3d93cbe7e9d1 None None] MFA Rules not processed for user `0d93485016824928ab7d2550b290c0c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:22:04.932224 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-000ffc52-dbca-4a4b-8538-3d93cbe7e9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:22:05.007896 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1124/2268] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:22:04 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:22:05.020751 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b78d13-7dad-4a5e-b349-f74782c5c6e2 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:22:05.022960 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-a1b78d13-7dad-4a5e-b349-f74782c5c6e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:22:05.052764 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:22:05.090685 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b78d13-7dad-4a5e-b349-f74782c5c6e2 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:22:05.095746 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca7a3a-8ba3-44a6-8dd7-5c1299d9957e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbaf66d6e1f54047917cc615546bbd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:22:05.096737 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca7a3a-8ba3-44a6-8dd7-5c1299d9957e service glance] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:22:05.096925 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca7a3a-8ba3-44a6-8dd7-5c1299d9957e service glance] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:22:05.097259 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca7a3a-8ba3-44a6-8dd7-5c1299d9957e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:22:05.167000 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca7a3a-8ba3-44a6-8dd7-5c1299d9957e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:22:05.168648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca7a3a-8ba3-44a6-8dd7-5c1299d9957e service glance] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:22:05.179210 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1145/2269] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:22:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 16:22:05.308840 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1bc2ab-c82c-4019-bbc0-e0371ef61a0b None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:22:05.309205 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1bc2ab-c82c-4019-bbc0-e0371ef61a0b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:22:05.309550 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1bc2ab-c82c-4019-bbc0-e0371ef61a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:22:05.328730 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-ee1bc2ab-c82c-4019-bbc0-e0371ef61a0b None None] MFA Rules not processed for user `875140cf51d9410da45f8c3805b26204`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:22:05.329995 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ee1bc2ab-c82c-4019-bbc0-e0371ef61a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:22:05.388848 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1125/2270] 10.222.0.41 () {58 vars in 1009 bytes} [Fri Jun 3 16:22:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:22:05.401889 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca7a3a-8ba3-44a6-8dd7-5c1299d9957e service glance] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:22:05.403389 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-4fca7a3a-8ba3-44a6-8dd7-5c1299d9957e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:22:05.482082 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca7a3a-8ba3-44a6-8dd7-5c1299d9957e service glance] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:22:05.489466 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474dabe-192d-4d9b-a838-99e3f24e96f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5f513b781a4a73b443cded13359eda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:22:05.490470 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474dabe-192d-4d9b-a838-99e3f24e96f5 service swift] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:22:05.490903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474dabe-192d-4d9b-a838-99e3f24e96f5 service swift] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:22:05.491358 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474dabe-192d-4d9b-a838-99e3f24e96f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:22:05.503758 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1474dabe-192d-4d9b-a838-99e3f24e96f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:22:05.506315 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1474dabe-192d-4d9b-a838-99e3f24e96f5 service swift] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:22:05.516621 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1146/2271] 10.222.0.41 () {60 vars in 1384 bytes} [Fri Jun 3 16:22:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 16:22:14.350802 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ecdca2-9442-4b7a-ae72-f957647e38d2 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:22:14.351503 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ecdca2-9442-4b7a-ae72-f957647e38d2 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:22:14.351503 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ecdca2-9442-4b7a-ae72-f957647e38d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:22:14.368649 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b3ecdca2-9442-4b7a-ae72-f957647e38d2 None None] MFA Rules not processed for user `1e415e30c70d441991187b13c229ee18`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:22:14.369835 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b3ecdca2-9442-4b7a-ae72-f957647e38d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:22:14.376580 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1126/2272] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:22:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:22:14.702706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c798c97-8949-4189-b12e-ca6a0a6817d3 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:22:14.703231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c798c97-8949-4189-b12e-ca6a0a6817d3 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:22:14.703577 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c798c97-8949-4189-b12e-ca6a0a6817d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:22:14.725535 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-5c798c97-8949-4189-b12e-ca6a0a6817d3 None None] MFA Rules not processed for user `875140cf51d9410da45f8c3805b26204`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:22:14.726915 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-5c798c97-8949-4189-b12e-ca6a0a6817d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:22:14.734263 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1147/2273] 10.222.0.41 () {58 vars in 967 bytes} [Fri Jun 3 16:22:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:22:14.747066 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ecdca2-9442-4b7a-ae72-f957647e38d2 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:22:14.759893 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ecdca2-9442-4b7a-ae72-f957647e38d2 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:22:14.764736 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff1ea58-6564-4889-95c8-446bec52f25e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5f513b781a4a73b443cded13359eda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:22:14.765377 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1ea58-6564-4889-95c8-446bec52f25e service swift] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:22:14.765660 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1ea58-6564-4889-95c8-446bec52f25e service swift] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:22:14.765936 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1ea58-6564-4889-95c8-446bec52f25e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:22:14.775376 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff1ea58-6564-4889-95c8-446bec52f25e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:22:14.776923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff1ea58-6564-4889-95c8-446bec52f25e service swift] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:22:14.786149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1127/2274] 10.222.0.41 () {60 vars in 1384 bytes} [Fri Jun 3 16:22:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 16:22:16.262127 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483e7cf-7a64-4461-b564-efaf4f5d5142 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:22:16.262550 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483e7cf-7a64-4461-b564-efaf4f5d5142 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:22:16.262829 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483e7cf-7a64-4461-b564-efaf4f5d5142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:22:16.281717 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-2483e7cf-7a64-4461-b564-efaf4f5d5142 None None] MFA Rules not processed for user `875140cf51d9410da45f8c3805b26204`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:22:16.282930 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-2483e7cf-7a64-4461-b564-efaf4f5d5142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:22:16.290583 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1148/2275] 10.222.0.41 () {58 vars in 1009 bytes} [Fri Jun 3 16:22:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:22:16.302600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff1ea58-6564-4889-95c8-446bec52f25e service swift] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:22:16.315267 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff1ea58-6564-4889-95c8-446bec52f25e service swift] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:22:16.321247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb347c-7bdd-40c2-b467-169b9e938de3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5f513b781a4a73b443cded13359eda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:22:16.321892 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb347c-7bdd-40c2-b467-169b9e938de3 service swift] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:22:16.322182 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb347c-7bdd-40c2-b467-169b9e938de3 service swift] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:22:16.322554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb347c-7bdd-40c2-b467-169b9e938de3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:22:16.331216 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bb347c-7bdd-40c2-b467-169b9e938de3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:22:16.332908 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bb347c-7bdd-40c2-b467-169b9e938de3 service swift] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:22:16.341687 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1128/2276] 10.222.0.41 () {60 vars in 1384 bytes} [Fri Jun 3 16:22:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 16:22:19.218540 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483e7cf-7a64-4461-b564-efaf4f5d5142 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:22:19.220223 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-2483e7cf-7a64-4461-b564-efaf4f5d5142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:22:19.285651 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:22:19.313030 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483e7cf-7a64-4461-b564-efaf4f5d5142 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:22:19.321289 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46235db-486d-4401-a58d-a6a6de538eee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:22:19.322059 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46235db-486d-4401-a58d-a6a6de538eee service nova] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:22:19.322418 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46235db-486d-4401-a58d-a6a6de538eee service nova] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:22:19.322722 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46235db-486d-4401-a58d-a6a6de538eee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:22:19.324528 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-f46235db-486d-4401-a58d-a6a6de538eee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:22:19.384958 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46235db-486d-4401-a58d-a6a6de538eee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:22:19.386472 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46235db-486d-4401-a58d-a6a6de538eee service nova] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:22:19.396520 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1149/2277] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:22:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:22:26.338409 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=72 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:22:29.391167 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=31 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:22:32.502714 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb347c-7bdd-40c2-b467-169b9e938de3 service swift] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:22:32.521494 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb347c-7bdd-40c2-b467-169b9e938de3 service swift] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:22:32.530707 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e5d63c-65db-424b-9ef5-2894bc058f55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:22:32.530707 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e5d63c-65db-424b-9ef5-2894bc058f55 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:22:32.531027 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e5d63c-65db-424b-9ef5-2894bc058f55 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:22:32.531114 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e5d63c-65db-424b-9ef5-2894bc058f55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:22:32.533007 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-65e5d63c-65db-424b-9ef5-2894bc058f55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:22:32.550061 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e5d63c-65db-424b-9ef5-2894bc058f55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:22:32.551695 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e5d63c-65db-424b-9ef5-2894bc058f55 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:22:32.563716 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1129/2278] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:22:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:22:42.559226 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=6 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:22:52.042888 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46235db-486d-4401-a58d-a6a6de538eee service nova] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:22:52.102087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46235db-486d-4401-a58d-a6a6de538eee service nova] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:22:52.105875 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c318fd04-79e7-498e-821b-6dfb6d99b399 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:22:52.106588 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c318fd04-79e7-498e-821b-6dfb6d99b399 service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:22:52.106900 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c318fd04-79e7-498e-821b-6dfb6d99b399 service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:22:52.107179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c318fd04-79e7-498e-821b-6dfb6d99b399 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:22:52.116138 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c318fd04-79e7-498e-821b-6dfb6d99b399 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:22:52.117554 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c318fd04-79e7-498e-821b-6dfb6d99b399 service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:22:52.126807 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1150/2279] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:22:52 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:23:02.123447 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=26 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:23:03.331660 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9c650-746b-4e65-972d-df40e027e941 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:23:03.332196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9c650-746b-4e65-972d-df40e027e941 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:23:03.332196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9c650-746b-4e65-972d-df40e027e941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:23:03.349109 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-75f9c650-746b-4e65-972d-df40e027e941 None None] MFA Rules not processed for user `4206a21c317e4b5aace8541699f8032d`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:23:03.350300 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-75f9c650-746b-4e65-972d-df40e027e941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:23:03.358556 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1130/2280] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:23:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:23:13.344139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:23:15.216984 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e29c8-0e1e-48af-a560-e37ce8b6767a None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:23:15.217435 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e29c8-0e1e-48af-a560-e37ce8b6767a None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:23:15.217527 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e29c8-0e1e-48af-a560-e37ce8b6767a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:23:15.235608 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-9e0e29c8-0e1e-48af-a560-e37ce8b6767a None None] MFA Rules not processed for user `331c97a12f1b4c2eb1eefa61056ce089`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:23:15.236962 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-9e0e29c8-0e1e-48af-a560-e37ce8b6767a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:23:15.296406 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1151/2281] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:23:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:23:25.289237 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=25 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:23:33.491114 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9c650-746b-4e65-972d-df40e027e941 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:23:33.492619 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-75f9c650-746b-4e65-972d-df40e027e941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:23:33.508764 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9c650-746b-4e65-972d-df40e027e941 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:23:33.514309 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a066d7-0069-4d56-8d55-9a563f253132 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:23:33.514982 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a066d7-0069-4d56-8d55-9a563f253132 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:23:33.515272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a066d7-0069-4d56-8d55-9a563f253132 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:23:33.515585 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a066d7-0069-4d56-8d55-9a563f253132 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:23:33.517405 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-78a066d7-0069-4d56-8d55-9a563f253132 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:23:33.529639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a066d7-0069-4d56-8d55-9a563f253132 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:23:33.531259 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a066d7-0069-4d56-8d55-9a563f253132 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:23:33.542089 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1131/2282] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:23:33 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:23:37.133913 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f48bfda-f5ba-4173-816f-081725b3f67f None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:23:37.134462 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f48bfda-f5ba-4173-816f-081725b3f67f None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:23:37.134644 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f48bfda-f5ba-4173-816f-081725b3f67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:23:37.154385 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-5f48bfda-f5ba-4173-816f-081725b3f67f None None] MFA Rules not processed for user `ef7c1e449b864b50b3c71c39f19968a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:23:37.155870 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-5f48bfda-f5ba-4173-816f-081725b3f67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:23:37.220765 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1152/2283] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:23:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:23:43.537615 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=6 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:23:47.213199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=25 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:23:49.429223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef78d4-e1a7-4fb4-a1f9-f5affc032760 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:23:49.429709 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef78d4-e1a7-4fb4-a1f9-f5affc032760 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:23:49.429807 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef78d4-e1a7-4fb4-a1f9-f5affc032760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:23:49.447913 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-fdef78d4-e1a7-4fb4-a1f9-f5affc032760 None None] MFA Rules not processed for user `94f91ae071b24e31a8ef8d50933ba4fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:23:49.449039 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-fdef78d4-e1a7-4fb4-a1f9-f5affc032760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:23:49.501651 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1132/2284] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:23:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:23:49.819137 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f48bfda-f5ba-4173-816f-081725b3f67f None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:23:49.821580 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-5f48bfda-f5ba-4173-816f-081725b3f67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:23:49.910212 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f48bfda-f5ba-4173-816f-081725b3f67f None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:23:49.915835 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ba4ec-94e6-4c4d-802e-423972a43ff3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be901786f8934d559e0bac5d622f97cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:23:49.916582 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ba4ec-94e6-4c4d-802e-423972a43ff3 service placement] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:23:49.917058 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ba4ec-94e6-4c4d-802e-423972a43ff3 service placement] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:23:49.917415 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ba4ec-94e6-4c4d-802e-423972a43ff3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:23:49.919348 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-fc4ba4ec-94e6-4c4d-802e-423972a43ff3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:23:49.933289 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ba4ec-94e6-4c4d-802e-423972a43ff3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:23:49.935720 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ba4ec-94e6-4c4d-802e-423972a43ff3 service placement] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:23:49.947235 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1153/2285] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:23:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:23:59.495062 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=25 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:23:59.942096 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=32 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:24:39.769785 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4405c2-ae0c-4695-a809-6bafa7efb4dc None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:24:39.770364 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4405c2-ae0c-4695-a809-6bafa7efb4dc None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:24:39.770505 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4405c2-ae0c-4695-a809-6bafa7efb4dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:24:39.788624 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-0b4405c2-ae0c-4695-a809-6bafa7efb4dc None None] MFA Rules not processed for user `c68189df1f9b4f9aacfa319381849697`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:24:39.789864 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0b4405c2-ae0c-4695-a809-6bafa7efb4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:24:39.845848 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1133/2286] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:24:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:24:48.970213 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ba4ec-94e6-4c4d-802e-423972a43ff3 service placement] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:24:48.971745 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-fc4ba4ec-94e6-4c4d-802e-423972a43ff3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:24:48.983783 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ba4ec-94e6-4c4d-802e-423972a43ff3 service placement] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:24:48.987366 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a573c6-e01a-4aad-85ce-c8e5b7ee56e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:24:48.988001 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a573c6-e01a-4aad-85ce-c8e5b7ee56e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:24:48.988292 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a573c6-e01a-4aad-85ce-c8e5b7ee56e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:24:48.988573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a573c6-e01a-4aad-85ce-c8e5b7ee56e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:24:48.990282 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-b9a573c6-e01a-4aad-85ce-c8e5b7ee56e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:24:49.052058 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a573c6-e01a-4aad-85ce-c8e5b7ee56e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:24:49.053477 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a573c6-e01a-4aad-85ce-c8e5b7ee56e3 service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:24:49.062858 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1154/2287] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:24:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:24:49.839268 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=25 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:24:52.361303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4405c2-ae0c-4695-a809-6bafa7efb4dc None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:24:52.376888 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4405c2-ae0c-4695-a809-6bafa7efb4dc None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:24:52.381943 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a1b90-1e4d-43d2-93ff-21d729fdf4a1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be901786f8934d559e0bac5d622f97cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:24:52.382583 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a1b90-1e4d-43d2-93ff-21d729fdf4a1 service placement] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:24:52.382874 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a1b90-1e4d-43d2-93ff-21d729fdf4a1 service placement] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:24:52.383355 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a1b90-1e4d-43d2-93ff-21d729fdf4a1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:24:52.385169 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a00a1b90-1e4d-43d2-93ff-21d729fdf4a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:24:52.397291 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00a1b90-1e4d-43d2-93ff-21d729fdf4a1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:24:52.398902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00a1b90-1e4d-43d2-93ff-21d729fdf4a1 service placement] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:24:52.411138 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1134/2288] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:24:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:24:52.426872 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a573c6-e01a-4aad-85ce-c8e5b7ee56e3 service neutron] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:24:52.428424 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-b9a573c6-e01a-4aad-85ce-c8e5b7ee56e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:24:52.441072 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a573c6-e01a-4aad-85ce-c8e5b7ee56e3 service neutron] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:24:52.447815 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998ef2a-c85f-4196-a05c-0a51d08bffdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:24:52.449064 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998ef2a-c85f-4196-a05c-0a51d08bffdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:24:52.449652 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998ef2a-c85f-4196-a05c-0a51d08bffdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:24:52.450193 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998ef2a-c85f-4196-a05c-0a51d08bffdd admin admin] PATH_INFO: `/v3/users/4206a21c317e4b5aace8541699f8032d` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:24:52.451425 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d998ef2a-c85f-4196-a05c-0a51d08bffdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=4206a21c317e4b5aace8541699f8032d)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:24:52.453357 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d998ef2a-c85f-4196-a05c-0a51d08bffdd admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:24:52.510390 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-d998ef2a-c85f-4196-a05c-0a51d08bffdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4206a21c317e4b5aace8541699f8032d'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:24:52.516980 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-d998ef2a-c85f-4196-a05c-0a51d08bffdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4206a21c317e4b5aace8541699f8032d'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:24:52.529716 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-d998ef2a-c85f-4196-a05c-0a51d08bffdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4206a21c317e4b5aace8541699f8032d'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:24:52.536796 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-d998ef2a-c85f-4196-a05c-0a51d08bffdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4206a21c317e4b5aace8541699f8032d'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:24:52.537415 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-d998ef2a-c85f-4196-a05c-0a51d08bffdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4206a21c317e4b5aace8541699f8032d'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:24:52.543230 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1155/2289] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:24:52 2022] DELETE /identity/v3/users/4206a21c317e4b5aace8541699f8032d => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:24:52.784067 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a1b90-1e4d-43d2-93ff-21d729fdf4a1 service placement] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:24:52.803934 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a1b90-1e4d-43d2-93ff-21d729fdf4a1 service placement] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:24:52.861946 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706be0d-1d50-40ae-b592-105ad8c6b112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:24:52.862821 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706be0d-1d50-40ae-b592-105ad8c6b112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:24:52.863110 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706be0d-1d50-40ae-b592-105ad8c6b112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:24:52.864051 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706be0d-1d50-40ae-b592-105ad8c6b112 admin admin] PATH_INFO: `/v3/projects/1443b802dc724950b5eb5c6446e2515c` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:24:52.865263 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9706be0d-1d50-40ae-b592-105ad8c6b112 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1443b802dc724950b5eb5c6446e2515c)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:24:52.867200 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9706be0d-1d50-40ae-b592-105ad8c6b112 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:24:52.919601 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9706be0d-1d50-40ae-b592-105ad8c6b112 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1443b802dc724950b5eb5c6446e2515c', 'request_id': 'req-9706be0d-1d50-40ae-b592-105ad8c6b112'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:24:52.926618 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9706be0d-1d50-40ae-b592-105ad8c6b112 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1443b802dc724950b5eb5c6446e2515c', 'request_id': 'req-9706be0d-1d50-40ae-b592-105ad8c6b112'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:24:52.933506 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9706be0d-1d50-40ae-b592-105ad8c6b112 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1443b802dc724950b5eb5c6446e2515c', 'request_id': 'req-9706be0d-1d50-40ae-b592-105ad8c6b112'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:24:52.946127 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9706be0d-1d50-40ae-b592-105ad8c6b112 admin admin] The token cache is being invalidate because project 1443b802dc724950b5eb5c6446e2515c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:24:52.946442 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9706be0d-1d50-40ae-b592-105ad8c6b112 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:24:52.947046 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9706be0d-1d50-40ae-b592-105ad8c6b112 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:24:52.948621 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1135/2290] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:24:52 2022] DELETE /identity/v3/projects/1443b802dc724950b5eb5c6446e2515c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:24:52.953549 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63254e3a-72ad-4239-bdc4-e67b95d01d6b None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:24:52.954871 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63254e3a-72ad-4239-bdc4-e67b95d01d6b None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:24:52.955527 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63254e3a-72ad-4239-bdc4-e67b95d01d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:24:52.974408 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-63254e3a-72ad-4239-bdc4-e67b95d01d6b None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:24:52.975796 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-63254e3a-72ad-4239-bdc4-e67b95d01d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:24:53.046626 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1156/2291] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:24:53.054580 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718cf3e1-5e92-4f3c-9325-9d8bbd100037 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:24:53.054889 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718cf3e1-5e92-4f3c-9325-9d8bbd100037 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:24:53.055173 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718cf3e1-5e92-4f3c-9325-9d8bbd100037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:24:53.073329 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-718cf3e1-5e92-4f3c-9325-9d8bbd100037 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:24:53.074688 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-718cf3e1-5e92-4f3c-9325-9d8bbd100037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:24:53.082818 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1136/2292] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:24:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:24:53.092330 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63254e3a-72ad-4239-bdc4-e67b95d01d6b None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:24:53.102769 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63254e3a-72ad-4239-bdc4-e67b95d01d6b None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:24:53.108560 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd7d33-fc9d-4491-b645-4aef2eda460b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:24:53.109341 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd7d33-fc9d-4491-b645-4aef2eda460b admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:24:53.109700 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd7d33-fc9d-4491-b645-4aef2eda460b admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:24:53.110049 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd7d33-fc9d-4491-b645-4aef2eda460b admin admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:24:53.110857 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dd7d33-fc9d-4491-b645-4aef2eda460b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:24:53.111183 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dd7d33-fc9d-4491-b645-4aef2eda460b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:24:53.112812 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dd7d33-fc9d-4491-b645-4aef2eda460b admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:24:53.115055 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:24:53.130554 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1157/2293] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:24:53.131445 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718cf3e1-5e92-4f3c-9325-9d8bbd100037 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:24:53.140796 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718cf3e1-5e92-4f3c-9325-9d8bbd100037 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:24:53.145789 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f111d07-16f1-4392-8232-8e00d0a90d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:24:53.146463 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f111d07-16f1-4392-8232-8e00d0a90d5c admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:24:53.146729 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f111d07-16f1-4392-8232-8e00d0a90d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:24:53.147004 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f111d07-16f1-4392-8232-8e00d0a90d5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:24:53.147734 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f111d07-16f1-4392-8232-8e00d0a90d5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:24:53.149675 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f111d07-16f1-4392-8232-8e00d0a90d5c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:24:53.183368 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1137/2294] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:24:53 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:24:53.184526 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd7d33-fc9d-4491-b645-4aef2eda460b admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:24:53.195383 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd7d33-fc9d-4491-b645-4aef2eda460b admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:24:53.258514 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b249781c-663b-42dd-91e8-0ce9ff5d9774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:24:53.259074 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249781c-663b-42dd-91e8-0ce9ff5d9774 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:24:53.259461 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249781c-663b-42dd-91e8-0ce9ff5d9774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:24:53.259813 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249781c-663b-42dd-91e8-0ce9ff5d9774 admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:24:53.260583 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b249781c-663b-42dd-91e8-0ce9ff5d9774 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:24:53.262745 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b249781c-663b-42dd-91e8-0ce9ff5d9774 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:24:53.289242 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1158/2295] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:24:53 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:24:53.293739 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f111d07-16f1-4392-8232-8e00d0a90d5c admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:24:53.304919 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f111d07-16f1-4392-8232-8e00d0a90d5c admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:24:53.309555 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c0c6e0-b2d8-40d2-8542-56f4f8e209d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:24:53.310226 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0c6e0-b2d8-40d2-8542-56f4f8e209d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:24:53.310514 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0c6e0-b2d8-40d2-8542-56f4f8e209d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:24:53.310797 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0c6e0-b2d8-40d2-8542-56f4f8e209d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:24:53.311567 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c0c6e0-b2d8-40d2-8542-56f4f8e209d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:24:53.313174 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c0c6e0-b2d8-40d2-8542-56f4f8e209d8 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:24:53.323667 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1138/2296] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:24:53.327798 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b249781c-663b-42dd-91e8-0ce9ff5d9774 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:24:53.338107 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b249781c-663b-42dd-91e8-0ce9ff5d9774 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:24:53.343599 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bb2a0d-a624-45b3-a6fd-dc576c604891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:24:53.344274 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb2a0d-a624-45b3-a6fd-dc576c604891 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:24:53.344568 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb2a0d-a624-45b3-a6fd-dc576c604891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:24:53.344845 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb2a0d-a624-45b3-a6fd-dc576c604891 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:24:53.345582 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bb2a0d-a624-45b3-a6fd-dc576c604891 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:24:53.347197 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bb2a0d-a624-45b3-a6fd-dc576c604891 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:24:53.362545 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1159/2297] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:24:53.364531 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c0c6e0-b2d8-40d2-8542-56f4f8e209d8 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:24:53.378914 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c0c6e0-b2d8-40d2-8542-56f4f8e209d8 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:24:53.383715 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783dae5c-c83a-4116-9d13-8b67897f4f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:24:53.384614 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783dae5c-c83a-4116-9d13-8b67897f4f8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:24:53.384899 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783dae5c-c83a-4116-9d13-8b67897f4f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:24:53.385174 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783dae5c-c83a-4116-9d13-8b67897f4f8e admin admin] PATH_INFO: `/v3/projects/e7fcd0ea340b433b9a2fc0907aedf434/users/aa76db5fc063438f865cb2e2909c8353/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:24:53.406101 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783dae5c-c83a-4116-9d13-8b67897f4f8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7fcd0ea340b433b9a2fc0907aedf434, user_id=aa76db5fc063438f865cb2e2909c8353, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:24:53.409223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783dae5c-c83a-4116-9d13-8b67897f4f8e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:24:53.430464 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1139/2298] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:24:53 2022] PUT /identity/v3/projects/e7fcd0ea340b433b9a2fc0907aedf434/users/aa76db5fc063438f865cb2e2909c8353/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:24:53.441091 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f7567-e3bb-4b87-97e1-bf97e5d368cd None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:24:53.442235 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f7567-e3bb-4b87-97e1-bf97e5d368cd None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:24:53.442235 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f7567-e3bb-4b87-97e1-bf97e5d368cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:24:53.474106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-fe1f7567-e3bb-4b87-97e1-bf97e5d368cd None None] MFA Rules not processed for user `aa76db5fc063438f865cb2e2909c8353`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:24:53.475120 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-fe1f7567-e3bb-4b87-97e1-bf97e5d368cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:24:53.545110 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1160/2299] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:24:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:24:53.556733 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783dae5c-c83a-4116-9d13-8b67897f4f8e admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:24:53.558446 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-783dae5c-c83a-4116-9d13-8b67897f4f8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:24:53.573810 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:24:53.636625 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783dae5c-c83a-4116-9d13-8b67897f4f8e admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:24:53.642805 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf9eb2-f585-456b-a565-1e703fca6fb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:24:53.643480 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf9eb2-f585-456b-a565-1e703fca6fb9 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:24:53.643766 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf9eb2-f585-456b-a565-1e703fca6fb9 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:24:53.644040 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf9eb2-f585-456b-a565-1e703fca6fb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:24:53.652499 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaf9eb2-f585-456b-a565-1e703fca6fb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:24:53.654082 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaf9eb2-f585-456b-a565-1e703fca6fb9 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:24:53.663857 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1140/2300] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:24:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:25:03.049580 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f7567-e3bb-4b87-97e1-bf97e5d368cd None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:03.109049 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f7567-e3bb-4b87-97e1-bf97e5d368cd None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:03.113521 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d38bd5-c848-4d34-998f-4e0987ed51c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:03.114391 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d38bd5-c848-4d34-998f-4e0987ed51c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:03.115161 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d38bd5-c848-4d34-998f-4e0987ed51c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:03.115480 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d38bd5-c848-4d34-998f-4e0987ed51c6 admin admin] PATH_INFO: `/v3/users/aa76db5fc063438f865cb2e2909c8353` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:03.116644 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d38bd5-c848-4d34-998f-4e0987ed51c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa76db5fc063438f865cb2e2909c8353)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:03.118542 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d38bd5-c848-4d34-998f-4e0987ed51c6 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:03.120833 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:25:03.164604 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-41d38bd5-c848-4d34-998f-4e0987ed51c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa76db5fc063438f865cb2e2909c8353'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:03.171677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-41d38bd5-c848-4d34-998f-4e0987ed51c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa76db5fc063438f865cb2e2909c8353'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:03.184986 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-41d38bd5-c848-4d34-998f-4e0987ed51c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa76db5fc063438f865cb2e2909c8353'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:03.192253 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-41d38bd5-c848-4d34-998f-4e0987ed51c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa76db5fc063438f865cb2e2909c8353'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:03.193509 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-41d38bd5-c848-4d34-998f-4e0987ed51c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa76db5fc063438f865cb2e2909c8353'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:03.199463 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1161/2301] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:25:03 2022] DELETE /identity/v3/users/aa76db5fc063438f865cb2e2909c8353 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:25:03.204003 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412e0ce-9b59-4490-aa90-403e58ba2b81 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:03.204512 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412e0ce-9b59-4490-aa90-403e58ba2b81 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:03.204633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412e0ce-9b59-4490-aa90-403e58ba2b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:03.221031 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-0412e0ce-9b59-4490-aa90-403e58ba2b81 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:25:03.222209 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0412e0ce-9b59-4490-aa90-403e58ba2b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:03.281181 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1141/2302] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:25:03.290907 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d38bd5-c848-4d34-998f-4e0987ed51c6 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:03.292413 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-41d38bd5-c848-4d34-998f-4e0987ed51c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:03.349424 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d38bd5-c848-4d34-998f-4e0987ed51c6 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:03.353694 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22344a2-1a8e-42c1-8a63-7e9e15d33362 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:03.354379 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22344a2-1a8e-42c1-8a63-7e9e15d33362 service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:03.354604 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22344a2-1a8e-42c1-8a63-7e9e15d33362 service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:03.354880 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22344a2-1a8e-42c1-8a63-7e9e15d33362 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:03.363998 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22344a2-1a8e-42c1-8a63-7e9e15d33362 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:03.365913 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22344a2-1a8e-42c1-8a63-7e9e15d33362 service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:03.375117 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1162/2303] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:25:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:25:03.633281 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412e0ce-9b59-4490-aa90-403e58ba2b81 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:03.634717 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0412e0ce-9b59-4490-aa90-403e58ba2b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:03.646552 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412e0ce-9b59-4490-aa90-403e58ba2b81 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:03.651143 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa9fbd-a2b4-4c41-92a6-1671952cae0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:03.652035 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa9fbd-a2b4-4c41-92a6-1671952cae0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:03.652317 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa9fbd-a2b4-4c41-92a6-1671952cae0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:03.652591 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa9fbd-a2b4-4c41-92a6-1671952cae0b admin admin] PATH_INFO: `/v3/projects/e7fcd0ea340b433b9a2fc0907aedf434` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:03.653768 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa9fbd-a2b4-4c41-92a6-1671952cae0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7fcd0ea340b433b9a2fc0907aedf434)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:03.655710 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa9fbd-a2b4-4c41-92a6-1671952cae0b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:03.702044 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1efa9fbd-a2b4-4c41-92a6-1671952cae0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7fcd0ea340b433b9a2fc0907aedf434', 'request_id': 'req-1efa9fbd-a2b4-4c41-92a6-1671952cae0b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:03.708024 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1efa9fbd-a2b4-4c41-92a6-1671952cae0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7fcd0ea340b433b9a2fc0907aedf434', 'request_id': 'req-1efa9fbd-a2b4-4c41-92a6-1671952cae0b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:03.715202 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1efa9fbd-a2b4-4c41-92a6-1671952cae0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7fcd0ea340b433b9a2fc0907aedf434', 'request_id': 'req-1efa9fbd-a2b4-4c41-92a6-1671952cae0b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:03.725873 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1efa9fbd-a2b4-4c41-92a6-1671952cae0b admin admin] The token cache is being invalidate because project e7fcd0ea340b433b9a2fc0907aedf434 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:25:03.726157 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1efa9fbd-a2b4-4c41-92a6-1671952cae0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:03.726774 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1efa9fbd-a2b4-4c41-92a6-1671952cae0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:03.728320 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1142/2304] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:25:03 2022] DELETE /identity/v3/projects/e7fcd0ea340b433b9a2fc0907aedf434 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:25:03.733325 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3500234-4322-41d5-9669-434baafc057a None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:03.733676 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3500234-4322-41d5-9669-434baafc057a None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:03.733959 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3500234-4322-41d5-9669-434baafc057a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:03.751738 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-b3500234-4322-41d5-9669-434baafc057a None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:25:03.752952 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-b3500234-4322-41d5-9669-434baafc057a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:03.810558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1163/2305] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:25:03.818107 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332349b-5437-472f-b995-fbc6fafe7113 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:03.818396 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332349b-5437-472f-b995-fbc6fafe7113 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:03.818676 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332349b-5437-472f-b995-fbc6fafe7113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:03.836480 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-d332349b-5437-472f-b995-fbc6fafe7113 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:25:03.837667 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d332349b-5437-472f-b995-fbc6fafe7113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:03.845353 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1143/2306] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:25:03.849850 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3500234-4322-41d5-9669-434baafc057a None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:03.859669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3500234-4322-41d5-9669-434baafc057a None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:03.864333 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f025e-e26f-4169-afef-80e483171e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:03.865610 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f025e-e26f-4169-afef-80e483171e65 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:03.865898 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f025e-e26f-4169-afef-80e483171e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:03.866178 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f025e-e26f-4169-afef-80e483171e65 admin admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:03.866907 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f025e-e26f-4169-afef-80e483171e65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:25:03.867219 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f025e-e26f-4169-afef-80e483171e65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:03.868883 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f025e-e26f-4169-afef-80e483171e65 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:03.880135 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1164/2307] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:25:03.884392 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332349b-5437-472f-b995-fbc6fafe7113 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:03.893569 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332349b-5437-472f-b995-fbc6fafe7113 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:03.898630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57a0bc-4f40-4e42-9b06-9440da86b4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:03.899263 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57a0bc-4f40-4e42-9b06-9440da86b4cc admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:03.899580 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57a0bc-4f40-4e42-9b06-9440da86b4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:03.899855 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57a0bc-4f40-4e42-9b06-9440da86b4cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:03.900548 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f57a0bc-4f40-4e42-9b06-9440da86b4cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:03.902425 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f57a0bc-4f40-4e42-9b06-9440da86b4cc admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:03.915604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:25:03.929280 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1144/2308] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:25:03 2022] POST /identity/v3/projects => generated 367 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:25:03.932603 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f025e-e26f-4169-afef-80e483171e65 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:03.942431 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f025e-e26f-4169-afef-80e483171e65 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:03.950231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:25:04.000303 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1717ad-1cc0-4196-abbe-74b9fbe71ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:04.000902 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1717ad-1cc0-4196-abbe-74b9fbe71ae2 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:04.001189 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1717ad-1cc0-4196-abbe-74b9fbe71ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:04.001468 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1717ad-1cc0-4196-abbe-74b9fbe71ae2 admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:04.002161 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1717ad-1cc0-4196-abbe-74b9fbe71ae2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:04.004093 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1717ad-1cc0-4196-abbe-74b9fbe71ae2 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:04.025151 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1165/2309] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:25:03 2022] POST /identity/v3/users => generated 284 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:25:04.029246 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57a0bc-4f40-4e42-9b06-9440da86b4cc admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:04.040206 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57a0bc-4f40-4e42-9b06-9440da86b4cc admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:04.044804 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb77f75-3698-4b9f-a591-460c691fa811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:04.045455 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb77f75-3698-4b9f-a591-460c691fa811 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:04.045743 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb77f75-3698-4b9f-a591-460c691fa811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:04.046023 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb77f75-3698-4b9f-a591-460c691fa811 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:04.046755 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb77f75-3698-4b9f-a591-460c691fa811 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:04.048460 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb77f75-3698-4b9f-a591-460c691fa811 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:04.058645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1145/2310] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:25:04.062547 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1717ad-1cc0-4196-abbe-74b9fbe71ae2 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:04.072124 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1717ad-1cc0-4196-abbe-74b9fbe71ae2 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:04.076615 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79672dad-64c0-41f7-8932-61911d3b8f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:04.077259 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79672dad-64c0-41f7-8932-61911d3b8f16 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:04.077548 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79672dad-64c0-41f7-8932-61911d3b8f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:04.077826 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79672dad-64c0-41f7-8932-61911d3b8f16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:04.078543 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79672dad-64c0-41f7-8932-61911d3b8f16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:04.080150 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79672dad-64c0-41f7-8932-61911d3b8f16 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:04.090171 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1166/2311] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:25:04.094041 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb77f75-3698-4b9f-a591-460c691fa811 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:04.104496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb77f75-3698-4b9f-a591-460c691fa811 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:04.109350 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d267cbe8-9b13-4070-8c6d-5c6f0fe63f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:04.110232 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267cbe8-9b13-4070-8c6d-5c6f0fe63f93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:04.110519 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267cbe8-9b13-4070-8c6d-5c6f0fe63f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:04.110798 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267cbe8-9b13-4070-8c6d-5c6f0fe63f93 admin admin] PATH_INFO: `/v3/projects/da237f964628428dade210cb860c6862/users/8cf211bc8beb4b7caffb26aa1d83400b/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:04.125027 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d267cbe8-9b13-4070-8c6d-5c6f0fe63f93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da237f964628428dade210cb860c6862, user_id=8cf211bc8beb4b7caffb26aa1d83400b, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:04.127721 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d267cbe8-9b13-4070-8c6d-5c6f0fe63f93 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:04.140304 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1146/2312] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:25:04 2022] PUT /identity/v3/projects/da237f964628428dade210cb860c6862/users/8cf211bc8beb4b7caffb26aa1d83400b/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:25:04.152237 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f42cfe-bf83-4c7f-ad61-894d686bcffe None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:04.152530 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f42cfe-bf83-4c7f-ad61-894d686bcffe None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:04.152815 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f42cfe-bf83-4c7f-ad61-894d686bcffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:04.179127 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-b4f42cfe-bf83-4c7f-ad61-894d686bcffe None None] MFA Rules not processed for user `8cf211bc8beb4b7caffb26aa1d83400b`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:25:04.180373 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-b4f42cfe-bf83-4c7f-ad61-894d686bcffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:04.243915 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1167/2313] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:25:04 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:25:04.254769 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d267cbe8-9b13-4070-8c6d-5c6f0fe63f93 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:04.257170 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d267cbe8-9b13-4070-8c6d-5c6f0fe63f93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:04.312368 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d267cbe8-9b13-4070-8c6d-5c6f0fe63f93 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:04.315992 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e61b1-0e39-4ea4-8f73-9e1ed853e886 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:04.316607 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e61b1-0e39-4ea4-8f73-9e1ed853e886 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:04.316887 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e61b1-0e39-4ea4-8f73-9e1ed853e886 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:04.317161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e61b1-0e39-4ea4-8f73-9e1ed853e886 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:04.325689 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56e61b1-0e39-4ea4-8f73-9e1ed853e886 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:04.327107 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56e61b1-0e39-4ea4-8f73-9e1ed853e886 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:04.336248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1147/2314] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:25:04 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:25:12.575801 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f42cfe-bf83-4c7f-ad61-894d686bcffe None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:12.637012 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f42cfe-bf83-4c7f-ad61-894d686bcffe None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:12.641515 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d87ec3-fb84-434e-99ee-0121ad2cf903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:12.642382 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d87ec3-fb84-434e-99ee-0121ad2cf903 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:12.642912 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d87ec3-fb84-434e-99ee-0121ad2cf903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:12.643198 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d87ec3-fb84-434e-99ee-0121ad2cf903 admin admin] PATH_INFO: `/v3/users/8cf211bc8beb4b7caffb26aa1d83400b` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:12.644417 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d87ec3-fb84-434e-99ee-0121ad2cf903 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cf211bc8beb4b7caffb26aa1d83400b)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:12.647014 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d87ec3-fb84-434e-99ee-0121ad2cf903 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:12.667984 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:25:12.702063 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-33d87ec3-fb84-434e-99ee-0121ad2cf903 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cf211bc8beb4b7caffb26aa1d83400b'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:12.709037 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-33d87ec3-fb84-434e-99ee-0121ad2cf903 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cf211bc8beb4b7caffb26aa1d83400b'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:12.721388 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-33d87ec3-fb84-434e-99ee-0121ad2cf903 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cf211bc8beb4b7caffb26aa1d83400b'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:12.728037 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-33d87ec3-fb84-434e-99ee-0121ad2cf903 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cf211bc8beb4b7caffb26aa1d83400b'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:12.728665 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-33d87ec3-fb84-434e-99ee-0121ad2cf903 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cf211bc8beb4b7caffb26aa1d83400b'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:12.734443 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1168/2315] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:25:12 2022] DELETE /identity/v3/users/8cf211bc8beb4b7caffb26aa1d83400b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:25:12.738932 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70557b22-900b-42b4-afec-48c2ecc99a68 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:12.739429 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70557b22-900b-42b4-afec-48c2ecc99a68 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:12.739550 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70557b22-900b-42b4-afec-48c2ecc99a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:12.756958 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-70557b22-900b-42b4-afec-48c2ecc99a68 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:25:12.758170 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-70557b22-900b-42b4-afec-48c2ecc99a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:12.815441 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1148/2316] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:25:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:25:12.825400 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d87ec3-fb84-434e-99ee-0121ad2cf903 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:12.826778 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-33d87ec3-fb84-434e-99ee-0121ad2cf903 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:12.883391 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d87ec3-fb84-434e-99ee-0121ad2cf903 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:12.887000 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd72ee89-ca14-40f2-8320-162f33ddb582 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:12.887668 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72ee89-ca14-40f2-8320-162f33ddb582 service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:12.887954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72ee89-ca14-40f2-8320-162f33ddb582 service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:12.888232 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72ee89-ca14-40f2-8320-162f33ddb582 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:12.897301 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd72ee89-ca14-40f2-8320-162f33ddb582 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:12.898717 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd72ee89-ca14-40f2-8320-162f33ddb582 service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:12.908277 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1169/2317] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:25:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:25:13.180967 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70557b22-900b-42b4-afec-48c2ecc99a68 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:13.182415 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-70557b22-900b-42b4-afec-48c2ecc99a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:13.194641 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70557b22-900b-42b4-afec-48c2ecc99a68 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:13.199561 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac6c53-0019-48b8-af63-85e655d73717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:13.200426 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac6c53-0019-48b8-af63-85e655d73717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:13.200712 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac6c53-0019-48b8-af63-85e655d73717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:13.200989 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac6c53-0019-48b8-af63-85e655d73717 admin admin] PATH_INFO: `/v3/projects/da237f964628428dade210cb860c6862` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:13.202207 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac6c53-0019-48b8-af63-85e655d73717 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da237f964628428dade210cb860c6862)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:13.204167 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac6c53-0019-48b8-af63-85e655d73717 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:13.251780 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-baac6c53-0019-48b8-af63-85e655d73717 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da237f964628428dade210cb860c6862', 'request_id': 'req-baac6c53-0019-48b8-af63-85e655d73717'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:13.258530 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-baac6c53-0019-48b8-af63-85e655d73717 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da237f964628428dade210cb860c6862', 'request_id': 'req-baac6c53-0019-48b8-af63-85e655d73717'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:13.261049 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:INSERT=3,keystone:SELECT=93,keystone:DELETE=4 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:25:13.266109 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-baac6c53-0019-48b8-af63-85e655d73717 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da237f964628428dade210cb860c6862', 'request_id': 'req-baac6c53-0019-48b8-af63-85e655d73717'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:13.277937 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-baac6c53-0019-48b8-af63-85e655d73717 admin admin] The token cache is being invalidate because project da237f964628428dade210cb860c6862 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:25:13.278229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-baac6c53-0019-48b8-af63-85e655d73717 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:13.278840 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-baac6c53-0019-48b8-af63-85e655d73717 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:13.280398 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1149/2318] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:25:13 2022] DELETE /identity/v3/projects/da237f964628428dade210cb860c6862 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:25:13.285267 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a984ed7f-cec9-4570-9552-277b8394d950 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:13.285561 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a984ed7f-cec9-4570-9552-277b8394d950 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:13.285840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a984ed7f-cec9-4570-9552-277b8394d950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:13.303661 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-a984ed7f-cec9-4570-9552-277b8394d950 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:25:13.304887 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-a984ed7f-cec9-4570-9552-277b8394d950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:13.361554 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1170/2319] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:25:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:25:13.369154 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1be26-9547-4507-acca-b28b027f19fa None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:13.369626 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1be26-9547-4507-acca-b28b027f19fa None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:13.369917 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1be26-9547-4507-acca-b28b027f19fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:13.388612 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-bac1be26-9547-4507-acca-b28b027f19fa None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:25:13.389822 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-bac1be26-9547-4507-acca-b28b027f19fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:13.398623 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1150/2320] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:25:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:25:13.402790 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a984ed7f-cec9-4570-9552-277b8394d950 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:13.412494 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a984ed7f-cec9-4570-9552-277b8394d950 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:13.417143 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965688e4-b1d5-4964-b59b-cc1620ff0795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:13.417802 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965688e4-b1d5-4964-b59b-cc1620ff0795 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:13.418094 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965688e4-b1d5-4964-b59b-cc1620ff0795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:13.418371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965688e4-b1d5-4964-b59b-cc1620ff0795 admin admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:13.419101 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965688e4-b1d5-4964-b59b-cc1620ff0795 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:25:13.419496 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965688e4-b1d5-4964-b59b-cc1620ff0795 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:13.421117 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965688e4-b1d5-4964-b59b-cc1620ff0795 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:13.432510 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1171/2321] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:25:13.436530 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac1be26-9547-4507-acca-b28b027f19fa None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:13.446212 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac1be26-9547-4507-acca-b28b027f19fa None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:13.450883 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f6b42-81eb-4e70-a348-797d7f473620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:13.451571 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f6b42-81eb-4e70-a348-797d7f473620 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:13.451858 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f6b42-81eb-4e70-a348-797d7f473620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:13.452135 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f6b42-81eb-4e70-a348-797d7f473620 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:13.452852 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f6b42-81eb-4e70-a348-797d7f473620 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:13.454755 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f6b42-81eb-4e70-a348-797d7f473620 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:13.480803 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1151/2322] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:25:13 2022] POST /identity/v3/projects => generated 393 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:25:13.484999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965688e4-b1d5-4964-b59b-cc1620ff0795 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:13.494920 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965688e4-b1d5-4964-b59b-cc1620ff0795 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:13.522842 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:25:13.554109 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074298b9-69e5-47f1-8e5e-9484e2801e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:13.554781 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074298b9-69e5-47f1-8e5e-9484e2801e20 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:13.555068 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074298b9-69e5-47f1-8e5e-9484e2801e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:13.555406 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074298b9-69e5-47f1-8e5e-9484e2801e20 admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:13.556094 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074298b9-69e5-47f1-8e5e-9484e2801e20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:13.558006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074298b9-69e5-47f1-8e5e-9484e2801e20 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:13.577383 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1172/2323] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:25:13 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:25:13.581506 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f6b42-81eb-4e70-a348-797d7f473620 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:13.592667 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f6b42-81eb-4e70-a348-797d7f473620 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:13.597687 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170639ff-c69f-49b0-855e-fcce36f2e9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:13.598347 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170639ff-c69f-49b0-855e-fcce36f2e9d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:13.598633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170639ff-c69f-49b0-855e-fcce36f2e9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:13.598911 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170639ff-c69f-49b0-855e-fcce36f2e9d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:13.599686 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170639ff-c69f-49b0-855e-fcce36f2e9d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:13.601283 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170639ff-c69f-49b0-855e-fcce36f2e9d3 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:13.611607 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1152/2324] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:25:13.615901 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074298b9-69e5-47f1-8e5e-9484e2801e20 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:13.625883 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074298b9-69e5-47f1-8e5e-9484e2801e20 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:13.630567 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef0cc93-2bdb-46c8-a405-c316c81fdc60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:13.631922 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0cc93-2bdb-46c8-a405-c316c81fdc60 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:13.632211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0cc93-2bdb-46c8-a405-c316c81fdc60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:13.632492 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0cc93-2bdb-46c8-a405-c316c81fdc60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:13.633212 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef0cc93-2bdb-46c8-a405-c316c81fdc60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:13.634803 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef0cc93-2bdb-46c8-a405-c316c81fdc60 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:13.645199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1173/2325] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:25:13.649018 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170639ff-c69f-49b0-855e-fcce36f2e9d3 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:13.658600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170639ff-c69f-49b0-855e-fcce36f2e9d3 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:13.663143 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81661c70-8951-4e7e-97a2-1de8cf38f53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:13.664036 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81661c70-8951-4e7e-97a2-1de8cf38f53d admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:13.664320 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81661c70-8951-4e7e-97a2-1de8cf38f53d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:13.664598 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81661c70-8951-4e7e-97a2-1de8cf38f53d admin admin] PATH_INFO: `/v3/projects/c7266cc526f34f4980c947e5e26cecbe/users/b61a1b9b38de4a85a759fb366420538b/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:13.680024 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81661c70-8951-4e7e-97a2-1de8cf38f53d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7266cc526f34f4980c947e5e26cecbe, user_id=b61a1b9b38de4a85a759fb366420538b, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:13.682622 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81661c70-8951-4e7e-97a2-1de8cf38f53d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:13.695590 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1153/2326] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:25:13 2022] PUT /identity/v3/projects/c7266cc526f34f4980c947e5e26cecbe/users/b61a1b9b38de4a85a759fb366420538b/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:25:13.705710 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3ba77-2160-4199-a132-3003c23cbb60 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:13.706008 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3ba77-2160-4199-a132-3003c23cbb60 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:13.706315 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3ba77-2160-4199-a132-3003c23cbb60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:13.732736 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-bee3ba77-2160-4199-a132-3003c23cbb60 None None] MFA Rules not processed for user `b61a1b9b38de4a85a759fb366420538b`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:25:13.733927 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-bee3ba77-2160-4199-a132-3003c23cbb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:13.796264 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1174/2327] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:25:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:25:13.800742 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81661c70-8951-4e7e-97a2-1de8cf38f53d admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:13.861068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81661c70-8951-4e7e-97a2-1de8cf38f53d admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:13.866492 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc6c176-72a4-4adf-8489-a40612b18d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:13.867072 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc6c176-72a4-4adf-8489-a40612b18d37 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:13.867384 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc6c176-72a4-4adf-8489-a40612b18d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:13.867662 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc6c176-72a4-4adf-8489-a40612b18d37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:13.868353 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc6c176-72a4-4adf-8489-a40612b18d37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:13.870238 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc6c176-72a4-4adf-8489-a40612b18d37 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:13.894676 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1154/2328] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:25:13 2022] POST /identity/v3/projects => generated 393 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:25:13.898937 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3ba77-2160-4199-a132-3003c23cbb60 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:13.909537 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3ba77-2160-4199-a132-3003c23cbb60 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:13.964499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca37850-be2e-48da-9c11-322ac381d86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:13.965172 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca37850-be2e-48da-9c11-322ac381d86d admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:13.965455 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca37850-be2e-48da-9c11-322ac381d86d admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:13.965736 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca37850-be2e-48da-9c11-322ac381d86d admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:13.966434 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca37850-be2e-48da-9c11-322ac381d86d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:13.968431 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca37850-be2e-48da-9c11-322ac381d86d admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:13.988288 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1175/2329] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:25:13 2022] POST /identity/v3/users => generated 303 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:25:13.992968 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc6c176-72a4-4adf-8489-a40612b18d37 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:14.003751 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc6c176-72a4-4adf-8489-a40612b18d37 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:14.008528 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21999d-1f3b-4168-9b86-f3d8fff5bf00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:14.009167 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21999d-1f3b-4168-9b86-f3d8fff5bf00 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:14.009449 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21999d-1f3b-4168-9b86-f3d8fff5bf00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:14.009721 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21999d-1f3b-4168-9b86-f3d8fff5bf00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:14.010443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc21999d-1f3b-4168-9b86-f3d8fff5bf00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:14.012082 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc21999d-1f3b-4168-9b86-f3d8fff5bf00 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:14.022584 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1155/2330] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:25:14.026533 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca37850-be2e-48da-9c11-322ac381d86d admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:14.036190 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca37850-be2e-48da-9c11-322ac381d86d admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:14.041226 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aef4af-d2ce-4d30-a661-1d1305372a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:14.042091 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aef4af-d2ce-4d30-a661-1d1305372a09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:14.042368 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aef4af-d2ce-4d30-a661-1d1305372a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:14.042641 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aef4af-d2ce-4d30-a661-1d1305372a09 admin admin] PATH_INFO: `/v3/projects/7bd581d6c52a42119fb776d0918eecc7/users/be42d14813014b42ae4f393033cd5e17/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:14.057363 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aef4af-d2ce-4d30-a661-1d1305372a09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bd581d6c52a42119fb776d0918eecc7, user_id=be42d14813014b42ae4f393033cd5e17, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:14.060044 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aef4af-d2ce-4d30-a661-1d1305372a09 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:14.072942 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1176/2331] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:25:14 2022] PUT /identity/v3/projects/7bd581d6c52a42119fb776d0918eecc7/users/be42d14813014b42ae4f393033cd5e17/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:25:14.076794 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21999d-1f3b-4168-9b86-f3d8fff5bf00 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:14.086638 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21999d-1f3b-4168-9b86-f3d8fff5bf00 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:14.135449 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:25:14.143404 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b2cd5-415a-4829-9b66-de67c61febf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:14.144287 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b2cd5-415a-4829-9b66-de67c61febf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:14.144744 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b2cd5-415a-4829-9b66-de67c61febf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:14.145081 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b2cd5-415a-4829-9b66-de67c61febf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:14.145967 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b2cd5-415a-4829-9b66-de67c61febf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:14.147666 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b2cd5-415a-4829-9b66-de67c61febf5 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:14.157221 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1156/2332] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:25:14.161239 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aef4af-d2ce-4d30-a661-1d1305372a09 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:14.167358 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:25:14.173557 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aef4af-d2ce-4d30-a661-1d1305372a09 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:14.179097 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a599ecdc-a61a-4914-b560-ab8f75e6b116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:14.180011 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599ecdc-a61a-4914-b560-ab8f75e6b116 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:14.180297 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599ecdc-a61a-4914-b560-ab8f75e6b116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:14.180573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599ecdc-a61a-4914-b560-ab8f75e6b116 admin admin] PATH_INFO: `/v3/projects/7bd581d6c52a42119fb776d0918eecc7/users/be42d14813014b42ae4f393033cd5e17/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:14.182559 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a599ecdc-a61a-4914-b560-ab8f75e6b116 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bd581d6c52a42119fb776d0918eecc7, user_id=be42d14813014b42ae4f393033cd5e17, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:14.185233 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a599ecdc-a61a-4914-b560-ab8f75e6b116 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:14.195358 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1177/2333] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:25:14 2022] PUT /identity/v3/projects/7bd581d6c52a42119fb776d0918eecc7/users/be42d14813014b42ae4f393033cd5e17/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:25:14.206483 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6471c3-d77c-42c2-b70a-de6af1c91b9d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:14.206775 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6471c3-d77c-42c2-b70a-de6af1c91b9d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:14.207059 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6471c3-d77c-42c2-b70a-de6af1c91b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:14.233058 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-3a6471c3-d77c-42c2-b70a-de6af1c91b9d None None] MFA Rules not processed for user `be42d14813014b42ae4f393033cd5e17`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:25:14.234262 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3a6471c3-d77c-42c2-b70a-de6af1c91b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:14.300983 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1157/2334] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:25:14 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:25:14.312024 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a599ecdc-a61a-4914-b560-ab8f75e6b116 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:14.313728 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-a599ecdc-a61a-4914-b560-ab8f75e6b116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:14.368251 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a599ecdc-a61a-4914-b560-ab8f75e6b116 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:14.371865 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff95e1-a464-4f2d-9381-94c518cfb503 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:14.372500 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff95e1-a464-4f2d-9381-94c518cfb503 service cinder] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:14.372784 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff95e1-a464-4f2d-9381-94c518cfb503 service cinder] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:14.373063 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff95e1-a464-4f2d-9381-94c518cfb503 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:14.381877 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff95e1-a464-4f2d-9381-94c518cfb503 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:14.383303 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff95e1-a464-4f2d-9381-94c518cfb503 service cinder] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:14.392510 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1178/2335] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:25:14 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:25:14.438048 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6471c3-d77c-42c2-b70a-de6af1c91b9d None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:14.439496 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3a6471c3-d77c-42c2-b70a-de6af1c91b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:14.451169 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6471c3-d77c-42c2-b70a-de6af1c91b9d None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:14.454890 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8958bcf-b13b-43bb-a1da-4a8e6f49fb00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:14.455567 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8958bcf-b13b-43bb-a1da-4a8e6f49fb00 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:14.455851 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8958bcf-b13b-43bb-a1da-4a8e6f49fb00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:14.456130 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8958bcf-b13b-43bb-a1da-4a8e6f49fb00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:14.511985 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8958bcf-b13b-43bb-a1da-4a8e6f49fb00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:14.513423 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8958bcf-b13b-43bb-a1da-4a8e6f49fb00 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:14.522907 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1158/2336] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:25:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:25:17.736694 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff95e1-a464-4f2d-9381-94c518cfb503 service cinder] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:17.797428 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff95e1-a464-4f2d-9381-94c518cfb503 service cinder] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:17.801999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279da208-6ac4-4730-8016-81769931797a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:17.802874 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279da208-6ac4-4730-8016-81769931797a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:17.803727 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279da208-6ac4-4730-8016-81769931797a admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:17.804009 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279da208-6ac4-4730-8016-81769931797a admin admin] PATH_INFO: `/v3/users/b61a1b9b38de4a85a759fb366420538b` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:17.805180 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279da208-6ac4-4730-8016-81769931797a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b61a1b9b38de4a85a759fb366420538b)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:17.807076 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279da208-6ac4-4730-8016-81769931797a admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:17.849614 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-279da208-6ac4-4730-8016-81769931797a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b61a1b9b38de4a85a759fb366420538b'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:17.856871 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-279da208-6ac4-4730-8016-81769931797a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b61a1b9b38de4a85a759fb366420538b'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:17.868995 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-279da208-6ac4-4730-8016-81769931797a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b61a1b9b38de4a85a759fb366420538b'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:17.875664 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-279da208-6ac4-4730-8016-81769931797a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b61a1b9b38de4a85a759fb366420538b'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:17.876291 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-279da208-6ac4-4730-8016-81769931797a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b61a1b9b38de4a85a759fb366420538b'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:17.882032 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1179/2337] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:25:17 2022] DELETE /identity/v3/users/b61a1b9b38de4a85a759fb366420538b => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:25:17.886450 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538d967-d1b7-460b-b327-33af4b548d9e None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:17.886918 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538d967-d1b7-460b-b327-33af4b548d9e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:17.887037 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538d967-d1b7-460b-b327-33af4b548d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:17.904231 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-2538d967-d1b7-460b-b327-33af4b548d9e None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:25:17.905418 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2538d967-d1b7-460b-b327-33af4b548d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:17.962048 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1159/2338] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:25:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:25:17.971523 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279da208-6ac4-4730-8016-81769931797a admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:17.972909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-279da208-6ac4-4730-8016-81769931797a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:17.986685 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:25:18.031491 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279da208-6ac4-4730-8016-81769931797a admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:18.035199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba198f3-5bf9-4976-8331-70405098d57b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:18.035866 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba198f3-5bf9-4976-8331-70405098d57b service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:18.036150 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba198f3-5bf9-4976-8331-70405098d57b service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:18.036426 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba198f3-5bf9-4976-8331-70405098d57b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:18.045779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba198f3-5bf9-4976-8331-70405098d57b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:18.047181 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba198f3-5bf9-4976-8331-70405098d57b service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:18.057447 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1180/2339] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:25:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:25:18.315270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538d967-d1b7-460b-b327-33af4b548d9e None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:18.316909 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2538d967-d1b7-460b-b327-33af4b548d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:18.331087 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538d967-d1b7-460b-b327-33af4b548d9e None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:18.337441 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761033e8-562c-4b69-8f38-175b7de1f80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:18.338481 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761033e8-562c-4b69-8f38-175b7de1f80d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:18.338889 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761033e8-562c-4b69-8f38-175b7de1f80d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:18.339310 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761033e8-562c-4b69-8f38-175b7de1f80d admin admin] PATH_INFO: `/v3/projects/c7266cc526f34f4980c947e5e26cecbe` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:18.340592 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761033e8-562c-4b69-8f38-175b7de1f80d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7266cc526f34f4980c947e5e26cecbe)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:18.342387 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761033e8-562c-4b69-8f38-175b7de1f80d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:18.353416 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:25:18.387635 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-761033e8-562c-4b69-8f38-175b7de1f80d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7266cc526f34f4980c947e5e26cecbe', 'request_id': 'req-761033e8-562c-4b69-8f38-175b7de1f80d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:18.393871 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-761033e8-562c-4b69-8f38-175b7de1f80d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7266cc526f34f4980c947e5e26cecbe', 'request_id': 'req-761033e8-562c-4b69-8f38-175b7de1f80d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:18.401825 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-761033e8-562c-4b69-8f38-175b7de1f80d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7266cc526f34f4980c947e5e26cecbe', 'request_id': 'req-761033e8-562c-4b69-8f38-175b7de1f80d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:18.412012 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-761033e8-562c-4b69-8f38-175b7de1f80d admin admin] The token cache is being invalidate because project c7266cc526f34f4980c947e5e26cecbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:25:18.412415 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-761033e8-562c-4b69-8f38-175b7de1f80d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:18.413514 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-761033e8-562c-4b69-8f38-175b7de1f80d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:18.415010 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1160/2340] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:25:18 2022] DELETE /identity/v3/projects/c7266cc526f34f4980c947e5e26cecbe => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:25:18.418959 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba198f3-5bf9-4976-8331-70405098d57b service neutron] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:18.428276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba198f3-5bf9-4976-8331-70405098d57b service neutron] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:18.481779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7458e5-952a-4d00-83c6-d620bb786a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:18.482638 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7458e5-952a-4d00-83c6-d620bb786a69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:18.482918 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7458e5-952a-4d00-83c6-d620bb786a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:18.483199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7458e5-952a-4d00-83c6-d620bb786a69 admin admin] PATH_INFO: `/v3/users/be42d14813014b42ae4f393033cd5e17` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:18.484385 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7458e5-952a-4d00-83c6-d620bb786a69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be42d14813014b42ae4f393033cd5e17)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:18.486264 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7458e5-952a-4d00-83c6-d620bb786a69 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:18.531941 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-9c7458e5-952a-4d00-83c6-d620bb786a69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be42d14813014b42ae4f393033cd5e17'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:18.538716 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-9c7458e5-952a-4d00-83c6-d620bb786a69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be42d14813014b42ae4f393033cd5e17'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:18.550242 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-9c7458e5-952a-4d00-83c6-d620bb786a69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be42d14813014b42ae4f393033cd5e17'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:18.556909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-9c7458e5-952a-4d00-83c6-d620bb786a69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be42d14813014b42ae4f393033cd5e17'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:18.557531 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-9c7458e5-952a-4d00-83c6-d620bb786a69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be42d14813014b42ae4f393033cd5e17'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:18.563262 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1181/2341] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:25:18 2022] DELETE /identity/v3/users/be42d14813014b42ae4f393033cd5e17 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:25:18.830243 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761033e8-562c-4b69-8f38-175b7de1f80d admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:18.831782 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-761033e8-562c-4b69-8f38-175b7de1f80d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:18.847258 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761033e8-562c-4b69-8f38-175b7de1f80d admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:18.903420 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe3344-80c2-46c2-b39e-f324c7dfce27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:18.904344 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe3344-80c2-46c2-b39e-f324c7dfce27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:18.904574 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe3344-80c2-46c2-b39e-f324c7dfce27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:18.904850 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe3344-80c2-46c2-b39e-f324c7dfce27 admin admin] PATH_INFO: `/v3/projects/7bd581d6c52a42119fb776d0918eecc7` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:18.906060 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe3344-80c2-46c2-b39e-f324c7dfce27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bd581d6c52a42119fb776d0918eecc7)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:18.908013 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe3344-80c2-46c2-b39e-f324c7dfce27 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:18.952422 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9ffe3344-80c2-46c2-b39e-f324c7dfce27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bd581d6c52a42119fb776d0918eecc7', 'request_id': 'req-9ffe3344-80c2-46c2-b39e-f324c7dfce27'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:18.958400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9ffe3344-80c2-46c2-b39e-f324c7dfce27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bd581d6c52a42119fb776d0918eecc7', 'request_id': 'req-9ffe3344-80c2-46c2-b39e-f324c7dfce27'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:18.966019 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9ffe3344-80c2-46c2-b39e-f324c7dfce27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bd581d6c52a42119fb776d0918eecc7', 'request_id': 'req-9ffe3344-80c2-46c2-b39e-f324c7dfce27'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:18.975972 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9ffe3344-80c2-46c2-b39e-f324c7dfce27 admin admin] The token cache is being invalidate because project 7bd581d6c52a42119fb776d0918eecc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:25:18.976257 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9ffe3344-80c2-46c2-b39e-f324c7dfce27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:18.976869 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9ffe3344-80c2-46c2-b39e-f324c7dfce27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:18.978378 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1161/2342] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:25:18 2022] DELETE /identity/v3/projects/7bd581d6c52a42119fb776d0918eecc7 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:25:18.983227 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2b111-f210-4f09-91f4-1aecaf5e5162 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:18.983697 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2b111-f210-4f09-91f4-1aecaf5e5162 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:18.983839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2b111-f210-4f09-91f4-1aecaf5e5162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:19.002047 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-40a2b111-f210-4f09-91f4-1aecaf5e5162 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:25:19.003248 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-40a2b111-f210-4f09-91f4-1aecaf5e5162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:19.031553 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:25:19.062738 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1182/2343] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:25:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:25:19.070622 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a275cad-b189-4603-86a2-121c86e2302d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:19.070915 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a275cad-b189-4603-86a2-121c86e2302d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:19.071196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a275cad-b189-4603-86a2-121c86e2302d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:19.089675 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-6a275cad-b189-4603-86a2-121c86e2302d None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:25:19.090894 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6a275cad-b189-4603-86a2-121c86e2302d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:19.098634 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1162/2344] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:25:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:25:19.103389 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2b111-f210-4f09-91f4-1aecaf5e5162 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:19.113236 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2b111-f210-4f09-91f4-1aecaf5e5162 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:19.118305 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f969b96f-fe72-4b8d-aab8-4341ccd9ee3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:19.118959 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969b96f-fe72-4b8d-aab8-4341ccd9ee3c admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:19.119244 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969b96f-fe72-4b8d-aab8-4341ccd9ee3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:19.119586 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969b96f-fe72-4b8d-aab8-4341ccd9ee3c admin admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:19.120306 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f969b96f-fe72-4b8d-aab8-4341ccd9ee3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:25:19.120617 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f969b96f-fe72-4b8d-aab8-4341ccd9ee3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:19.122216 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f969b96f-fe72-4b8d-aab8-4341ccd9ee3c admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:19.134116 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1183/2345] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:25:19.138234 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a275cad-b189-4603-86a2-121c86e2302d None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:19.147494 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a275cad-b189-4603-86a2-121c86e2302d None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:19.152027 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e90c4-04dd-46b8-8620-b289ce58718f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:19.152661 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e90c4-04dd-46b8-8620-b289ce58718f admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:19.152944 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e90c4-04dd-46b8-8620-b289ce58718f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:19.153218 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e90c4-04dd-46b8-8620-b289ce58718f admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:19.153899 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1e90c4-04dd-46b8-8620-b289ce58718f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:19.155895 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1e90c4-04dd-46b8-8620-b289ce58718f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:19.174287 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:25:19.182251 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1163/2346] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:25:19 2022] POST /identity/v3/projects => generated 381 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:25:19.186125 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f969b96f-fe72-4b8d-aab8-4341ccd9ee3c admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:19.196392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f969b96f-fe72-4b8d-aab8-4341ccd9ee3c admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:19.252246 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f07bd02-1211-49d5-aadc-5553942f3509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:19.253068 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07bd02-1211-49d5-aadc-5553942f3509 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:19.253225 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07bd02-1211-49d5-aadc-5553942f3509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:19.253501 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07bd02-1211-49d5-aadc-5553942f3509 admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:19.254226 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f07bd02-1211-49d5-aadc-5553942f3509 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:19.256739 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f07bd02-1211-49d5-aadc-5553942f3509 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:19.277369 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1184/2347] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:25:19 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:25:19.281412 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e90c4-04dd-46b8-8620-b289ce58718f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:19.291597 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e90c4-04dd-46b8-8620-b289ce58718f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:19.296188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a4ea3-4ef8-470e-a801-9597b7dcb139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:19.296846 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a4ea3-4ef8-470e-a801-9597b7dcb139 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:19.297130 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a4ea3-4ef8-470e-a801-9597b7dcb139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:19.297407 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a4ea3-4ef8-470e-a801-9597b7dcb139 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:19.298132 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5a4ea3-4ef8-470e-a801-9597b7dcb139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:19.299762 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5a4ea3-4ef8-470e-a801-9597b7dcb139 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:19.311739 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1164/2348] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:25:19.315621 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f07bd02-1211-49d5-aadc-5553942f3509 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:19.326524 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f07bd02-1211-49d5-aadc-5553942f3509 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:19.332495 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd02e22e-bbd2-4c1f-9824-f73933d72808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:19.333170 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd02e22e-bbd2-4c1f-9824-f73933d72808 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:19.333461 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd02e22e-bbd2-4c1f-9824-f73933d72808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:19.333743 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd02e22e-bbd2-4c1f-9824-f73933d72808 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:19.335141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd02e22e-bbd2-4c1f-9824-f73933d72808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:19.336774 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd02e22e-bbd2-4c1f-9824-f73933d72808 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:19.346836 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1185/2349] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:25:19.350952 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a4ea3-4ef8-470e-a801-9597b7dcb139 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:19.360801 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a4ea3-4ef8-470e-a801-9597b7dcb139 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:19.365610 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58570dcd-c4e9-48fc-83ae-704ae019e81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:19.366538 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58570dcd-c4e9-48fc-83ae-704ae019e81b admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:19.366823 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58570dcd-c4e9-48fc-83ae-704ae019e81b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:19.367100 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58570dcd-c4e9-48fc-83ae-704ae019e81b admin admin] PATH_INFO: `/v3/projects/aba62e22962d4e73bb1ae7e4769c71b5/users/132f1fa6064a43f4a88a3b4bb18b9d73/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:19.382076 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58570dcd-c4e9-48fc-83ae-704ae019e81b admin admin] RBAC: Authorizing `identity:create_grant(project_id=aba62e22962d4e73bb1ae7e4769c71b5, user_id=132f1fa6064a43f4a88a3b4bb18b9d73, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:19.384756 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58570dcd-c4e9-48fc-83ae-704ae019e81b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:19.398733 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1165/2350] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:25:19 2022] PUT /identity/v3/projects/aba62e22962d4e73bb1ae7e4769c71b5/users/132f1fa6064a43f4a88a3b4bb18b9d73/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:25:19.409874 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7fe1d-5aee-423f-a5d9-00b9a178d8dc None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:19.410120 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7fe1d-5aee-423f-a5d9-00b9a178d8dc None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:19.410421 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7fe1d-5aee-423f-a5d9-00b9a178d8dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:19.436466 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-58b7fe1d-5aee-423f-a5d9-00b9a178d8dc None None] MFA Rules not processed for user `132f1fa6064a43f4a88a3b4bb18b9d73`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:25:19.437671 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-58b7fe1d-5aee-423f-a5d9-00b9a178d8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:19.502549 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1186/2351] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:25:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:25:19.507470 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58570dcd-c4e9-48fc-83ae-704ae019e81b admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:19.566234 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58570dcd-c4e9-48fc-83ae-704ae019e81b admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:19.571404 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29bc077-5270-447c-8644-cb67251965f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:19.572043 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29bc077-5270-447c-8644-cb67251965f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:19.572327 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29bc077-5270-447c-8644-cb67251965f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:19.572602 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29bc077-5270-447c-8644-cb67251965f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:19.573286 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29bc077-5270-447c-8644-cb67251965f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:19.575166 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29bc077-5270-447c-8644-cb67251965f6 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:19.599273 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1166/2352] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:25:19 2022] POST /identity/v3/projects => generated 383 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:25:19.603682 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b7fe1d-5aee-423f-a5d9-00b9a178d8dc None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:19.614337 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b7fe1d-5aee-423f-a5d9-00b9a178d8dc None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:19.640707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:25:19.672409 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e2445-dcdb-4726-8306-485dafcf2acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:19.673002 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e2445-dcdb-4726-8306-485dafcf2acf admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:19.673290 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e2445-dcdb-4726-8306-485dafcf2acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:19.673567 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e2445-dcdb-4726-8306-485dafcf2acf admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:19.674247 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433e2445-dcdb-4726-8306-485dafcf2acf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:19.676213 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433e2445-dcdb-4726-8306-485dafcf2acf admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:19.696903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1187/2353] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:25:19 2022] POST /identity/v3/users => generated 298 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:25:19.700829 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29bc077-5270-447c-8644-cb67251965f6 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:19.711465 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29bc077-5270-447c-8644-cb67251965f6 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:19.716072 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd97f5b-be6b-4736-b806-c1505f460b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:19.716726 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd97f5b-be6b-4736-b806-c1505f460b62 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:19.717009 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd97f5b-be6b-4736-b806-c1505f460b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:19.717294 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd97f5b-be6b-4736-b806-c1505f460b62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:19.718022 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd97f5b-be6b-4736-b806-c1505f460b62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:19.719677 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd97f5b-be6b-4736-b806-c1505f460b62 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:19.729978 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1167/2354] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:25:19.734100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e2445-dcdb-4726-8306-485dafcf2acf admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:19.744700 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e2445-dcdb-4726-8306-485dafcf2acf admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:19.749335 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb620b-f5c5-447a-8381-609518ba57ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:19.750201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb620b-f5c5-447a-8381-609518ba57ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:19.750508 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb620b-f5c5-447a-8381-609518ba57ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:19.750785 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb620b-f5c5-447a-8381-609518ba57ab admin admin] PATH_INFO: `/v3/projects/f7e6f1965bff4481ab70c23b4972739d/users/2ab9d444cf714f899ca456c0c2ee97c6/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:19.765727 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bb620b-f5c5-447a-8381-609518ba57ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7e6f1965bff4481ab70c23b4972739d, user_id=2ab9d444cf714f899ca456c0c2ee97c6, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:19.768645 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bb620b-f5c5-447a-8381-609518ba57ab admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:19.781460 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1188/2355] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:25:19 2022] PUT /identity/v3/projects/f7e6f1965bff4481ab70c23b4972739d/users/2ab9d444cf714f899ca456c0c2ee97c6/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:25:19.785262 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd97f5b-be6b-4736-b806-c1505f460b62 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:19.795366 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd97f5b-be6b-4736-b806-c1505f460b62 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:19.852011 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35270622-c3eb-4f13-8a75-b1f727295ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:19.852676 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35270622-c3eb-4f13-8a75-b1f727295ca8 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:19.852966 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35270622-c3eb-4f13-8a75-b1f727295ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:19.853250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35270622-c3eb-4f13-8a75-b1f727295ca8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:19.853979 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35270622-c3eb-4f13-8a75-b1f727295ca8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:19.855613 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35270622-c3eb-4f13-8a75-b1f727295ca8 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:19.865130 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1168/2356] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:25:19.869749 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb620b-f5c5-447a-8381-609518ba57ab admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:19.879687 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb620b-f5c5-447a-8381-609518ba57ab admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:19.884857 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1872a17f-c116-487b-8b40-6b990123099a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:19.885727 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1872a17f-c116-487b-8b40-6b990123099a admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:19.886006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1872a17f-c116-487b-8b40-6b990123099a admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:19.886281 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1872a17f-c116-487b-8b40-6b990123099a admin admin] PATH_INFO: `/v3/projects/f7e6f1965bff4481ab70c23b4972739d/users/2ab9d444cf714f899ca456c0c2ee97c6/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:19.888240 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1872a17f-c116-487b-8b40-6b990123099a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7e6f1965bff4481ab70c23b4972739d, user_id=2ab9d444cf714f899ca456c0c2ee97c6, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:19.890815 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1872a17f-c116-487b-8b40-6b990123099a admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:19.901004 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1189/2357] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:25:19 2022] PUT /identity/v3/projects/f7e6f1965bff4481ab70c23b4972739d/users/2ab9d444cf714f899ca456c0c2ee97c6/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:25:19.910758 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939157a-3d54-4c24-803b-a3d87b935a43 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:19.911231 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939157a-3d54-4c24-803b-a3d87b935a43 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:19.911380 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939157a-3d54-4c24-803b-a3d87b935a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:19.937428 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-a939157a-3d54-4c24-803b-a3d87b935a43 None None] MFA Rules not processed for user `2ab9d444cf714f899ca456c0c2ee97c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:25:19.938567 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a939157a-3d54-4c24-803b-a3d87b935a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:19.961943 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:25:20.006475 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1169/2358] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:25:19 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:25:20.017640 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1872a17f-c116-487b-8b40-6b990123099a admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:20.019594 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-1872a17f-c116-487b-8b40-6b990123099a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:20.075307 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1872a17f-c116-487b-8b40-6b990123099a admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:20.079678 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e1cf8e-79a0-48a8-8c20-0a12e185f2d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:20.080321 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1cf8e-79a0-48a8-8c20-0a12e185f2d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:20.080601 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1cf8e-79a0-48a8-8c20-0a12e185f2d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:20.080874 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1cf8e-79a0-48a8-8c20-0a12e185f2d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:20.090231 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e1cf8e-79a0-48a8-8c20-0a12e185f2d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:20.091689 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e1cf8e-79a0-48a8-8c20-0a12e185f2d1 service cinder] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:20.101322 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1190/2359] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:25:20 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:25:20.194874 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939157a-3d54-4c24-803b-a3d87b935a43 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:20.204602 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939157a-3d54-4c24-803b-a3d87b935a43 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:20.208282 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d34a7-8212-4648-a732-8e411d2c5f25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:20.208921 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d34a7-8212-4648-a732-8e411d2c5f25 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:20.209233 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d34a7-8212-4648-a732-8e411d2c5f25 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:20.209508 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d34a7-8212-4648-a732-8e411d2c5f25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:20.265023 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0d34a7-8212-4648-a732-8e411d2c5f25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:20.266473 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0d34a7-8212-4648-a732-8e411d2c5f25 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:20.275488 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1170/2360] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:25:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:25:20.530161 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e1cf8e-79a0-48a8-8c20-0a12e185f2d1 service cinder] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:20.589633 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e1cf8e-79a0-48a8-8c20-0a12e185f2d1 service cinder] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:20.594178 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0250f854-0c0d-4928-bab8-bb28079e8f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:20.595040 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250f854-0c0d-4928-bab8-bb28079e8f8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:20.595343 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250f854-0c0d-4928-bab8-bb28079e8f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:20.595624 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250f854-0c0d-4928-bab8-bb28079e8f8f admin admin] PATH_INFO: `/v3/users/132f1fa6064a43f4a88a3b4bb18b9d73` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:20.596787 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0250f854-0c0d-4928-bab8-bb28079e8f8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=132f1fa6064a43f4a88a3b4bb18b9d73)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:20.598685 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0250f854-0c0d-4928-bab8-bb28079e8f8f admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:20.626207 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:25:20.645453 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0250f854-0c0d-4928-bab8-bb28079e8f8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '132f1fa6064a43f4a88a3b4bb18b9d73'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:20.652944 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0250f854-0c0d-4928-bab8-bb28079e8f8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '132f1fa6064a43f4a88a3b4bb18b9d73'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:20.665267 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0250f854-0c0d-4928-bab8-bb28079e8f8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '132f1fa6064a43f4a88a3b4bb18b9d73'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:20.671918 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0250f854-0c0d-4928-bab8-bb28079e8f8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '132f1fa6064a43f4a88a3b4bb18b9d73'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:20.672565 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0250f854-0c0d-4928-bab8-bb28079e8f8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '132f1fa6064a43f4a88a3b4bb18b9d73'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:20.678291 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1191/2361] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:25:20 2022] DELETE /identity/v3/users/132f1fa6064a43f4a88a3b4bb18b9d73 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:25:20.682853 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf990b-93ed-4659-8257-2882ad302526 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:20.683150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf990b-93ed-4659-8257-2882ad302526 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:20.683475 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf990b-93ed-4659-8257-2882ad302526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:20.700056 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-41bf990b-93ed-4659-8257-2882ad302526 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:25:20.701233 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-41bf990b-93ed-4659-8257-2882ad302526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:20.761104 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1171/2362] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:25:20.772698 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0250f854-0c0d-4928-bab8-bb28079e8f8f admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:20.774094 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-0250f854-0c0d-4928-bab8-bb28079e8f8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:20.829607 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0250f854-0c0d-4928-bab8-bb28079e8f8f admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:20.833482 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ca0ec-e4fc-40fa-a517-691609175a93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:20.834118 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ca0ec-e4fc-40fa-a517-691609175a93 service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:20.834973 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ca0ec-e4fc-40fa-a517-691609175a93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:20.835252 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ca0ec-e4fc-40fa-a517-691609175a93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:20.844087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6ca0ec-e4fc-40fa-a517-691609175a93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:20.845487 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6ca0ec-e4fc-40fa-a517-691609175a93 service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:20.854625 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1192/2363] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:25:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:25:21.121436 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf990b-93ed-4659-8257-2882ad302526 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:21.122982 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-41bf990b-93ed-4659-8257-2882ad302526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:21.135364 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf990b-93ed-4659-8257-2882ad302526 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:21.139984 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560495af-6791-4255-a4b5-f66922beb14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:21.140848 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560495af-6791-4255-a4b5-f66922beb14c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:21.141162 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560495af-6791-4255-a4b5-f66922beb14c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:21.141448 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560495af-6791-4255-a4b5-f66922beb14c admin admin] PATH_INFO: `/v3/projects/aba62e22962d4e73bb1ae7e4769c71b5` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:21.142650 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560495af-6791-4255-a4b5-f66922beb14c admin admin] RBAC: Authorizing `identity:delete_project(project_id=aba62e22962d4e73bb1ae7e4769c71b5)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:21.144947 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560495af-6791-4255-a4b5-f66922beb14c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:21.190019 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-560495af-6791-4255-a4b5-f66922beb14c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aba62e22962d4e73bb1ae7e4769c71b5', 'request_id': 'req-560495af-6791-4255-a4b5-f66922beb14c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:21.196811 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-560495af-6791-4255-a4b5-f66922beb14c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aba62e22962d4e73bb1ae7e4769c71b5', 'request_id': 'req-560495af-6791-4255-a4b5-f66922beb14c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:21.198437 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:25:21.205054 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-560495af-6791-4255-a4b5-f66922beb14c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aba62e22962d4e73bb1ae7e4769c71b5', 'request_id': 'req-560495af-6791-4255-a4b5-f66922beb14c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:21.216104 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-560495af-6791-4255-a4b5-f66922beb14c admin admin] The token cache is being invalidate because project aba62e22962d4e73bb1ae7e4769c71b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:25:21.216391 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-560495af-6791-4255-a4b5-f66922beb14c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:21.217013 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-560495af-6791-4255-a4b5-f66922beb14c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:21.218531 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1172/2364] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:25:21 2022] DELETE /identity/v3/projects/aba62e22962d4e73bb1ae7e4769c71b5 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:25:21.222507 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ca0ec-e4fc-40fa-a517-691609175a93 service neutron] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:21.232892 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ca0ec-e4fc-40fa-a517-691609175a93 service neutron] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:21.288666 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09acc785-052d-4bef-99bf-c3fd2acd71c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:21.289528 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09acc785-052d-4bef-99bf-c3fd2acd71c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:21.289809 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09acc785-052d-4bef-99bf-c3fd2acd71c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:21.290081 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09acc785-052d-4bef-99bf-c3fd2acd71c7 admin admin] PATH_INFO: `/v3/users/2ab9d444cf714f899ca456c0c2ee97c6` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:21.291428 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09acc785-052d-4bef-99bf-c3fd2acd71c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ab9d444cf714f899ca456c0c2ee97c6)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:21.293323 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09acc785-052d-4bef-99bf-c3fd2acd71c7 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:21.337515 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-09acc785-052d-4bef-99bf-c3fd2acd71c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ab9d444cf714f899ca456c0c2ee97c6'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:21.339165 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:25:21.347258 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-09acc785-052d-4bef-99bf-c3fd2acd71c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ab9d444cf714f899ca456c0c2ee97c6'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:21.361014 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-09acc785-052d-4bef-99bf-c3fd2acd71c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ab9d444cf714f899ca456c0c2ee97c6'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:21.367775 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-09acc785-052d-4bef-99bf-c3fd2acd71c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ab9d444cf714f899ca456c0c2ee97c6'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:21.368454 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-09acc785-052d-4bef-99bf-c3fd2acd71c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ab9d444cf714f899ca456c0c2ee97c6'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:21.374677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1193/2365] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:25:21 2022] DELETE /identity/v3/users/2ab9d444cf714f899ca456c0c2ee97c6 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:25:21.634712 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560495af-6791-4255-a4b5-f66922beb14c admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:21.636325 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-560495af-6791-4255-a4b5-f66922beb14c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:21.652510 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560495af-6791-4255-a4b5-f66922beb14c admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:21.708974 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a09cdd8-96db-484e-a7ad-f9c9ce339420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:21.710441 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09cdd8-96db-484e-a7ad-f9c9ce339420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:21.710721 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09cdd8-96db-484e-a7ad-f9c9ce339420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:21.710993 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09cdd8-96db-484e-a7ad-f9c9ce339420 admin admin] PATH_INFO: `/v3/projects/f7e6f1965bff4481ab70c23b4972739d` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:21.712187 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a09cdd8-96db-484e-a7ad-f9c9ce339420 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7e6f1965bff4481ab70c23b4972739d)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:21.714087 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a09cdd8-96db-484e-a7ad-f9c9ce339420 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:21.757424 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5a09cdd8-96db-484e-a7ad-f9c9ce339420 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7e6f1965bff4481ab70c23b4972739d', 'request_id': 'req-5a09cdd8-96db-484e-a7ad-f9c9ce339420'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:21.763196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5a09cdd8-96db-484e-a7ad-f9c9ce339420 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7e6f1965bff4481ab70c23b4972739d', 'request_id': 'req-5a09cdd8-96db-484e-a7ad-f9c9ce339420'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:21.770418 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5a09cdd8-96db-484e-a7ad-f9c9ce339420 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7e6f1965bff4481ab70c23b4972739d', 'request_id': 'req-5a09cdd8-96db-484e-a7ad-f9c9ce339420'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:21.781283 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5a09cdd8-96db-484e-a7ad-f9c9ce339420 admin admin] The token cache is being invalidate because project f7e6f1965bff4481ab70c23b4972739d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:25:21.781572 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5a09cdd8-96db-484e-a7ad-f9c9ce339420 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:21.782201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5a09cdd8-96db-484e-a7ad-f9c9ce339420 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:21.783760 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1173/2366] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:25:21 2022] DELETE /identity/v3/projects/f7e6f1965bff4481ab70c23b4972739d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:25:21.788799 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd55e54-e35a-4b18-b239-5a9a9f254219 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:21.789091 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd55e54-e35a-4b18-b239-5a9a9f254219 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:21.789369 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd55e54-e35a-4b18-b239-5a9a9f254219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:21.807097 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-ecd55e54-e35a-4b18-b239-5a9a9f254219 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:25:21.808406 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-ecd55e54-e35a-4b18-b239-5a9a9f254219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:21.866128 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1194/2367] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:25:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:25:21.873951 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06d03a-f128-4c20-abb5-2ab907a65d8d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:21.874253 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06d03a-f128-4c20-abb5-2ab907a65d8d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:21.874542 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06d03a-f128-4c20-abb5-2ab907a65d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:21.894170 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-af06d03a-f128-4c20-abb5-2ab907a65d8d None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:25:21.895404 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-af06d03a-f128-4c20-abb5-2ab907a65d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:21.903370 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1174/2368] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:25:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:25:21.907377 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd55e54-e35a-4b18-b239-5a9a9f254219 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:21.916940 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd55e54-e35a-4b18-b239-5a9a9f254219 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:21.921472 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f90ca-c11e-484e-b87f-0b1a874de567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:21.922118 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f90ca-c11e-484e-b87f-0b1a874de567 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:21.922406 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f90ca-c11e-484e-b87f-0b1a874de567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:21.922684 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f90ca-c11e-484e-b87f-0b1a874de567 admin admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:21.923436 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f90ca-c11e-484e-b87f-0b1a874de567 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:25:21.923746 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f90ca-c11e-484e-b87f-0b1a874de567 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:21.925321 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f90ca-c11e-484e-b87f-0b1a874de567 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:21.937026 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1195/2369] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:25:21.941171 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af06d03a-f128-4c20-abb5-2ab907a65d8d None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:21.950931 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af06d03a-f128-4c20-abb5-2ab907a65d8d None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:21.955571 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f849b-10ab-42fe-8c08-ff19b0e29558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:21.956209 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f849b-10ab-42fe-8c08-ff19b0e29558 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:21.956491 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f849b-10ab-42fe-8c08-ff19b0e29558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:21.956764 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f849b-10ab-42fe-8c08-ff19b0e29558 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:21.957461 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018f849b-10ab-42fe-8c08-ff19b0e29558 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:21.959401 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018f849b-10ab-42fe-8c08-ff19b0e29558 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:21.983172 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1175/2370] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:25:21 2022] POST /identity/v3/projects => generated 383 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:25:21.987244 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f90ca-c11e-484e-b87f-0b1a874de567 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:21.997617 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f90ca-c11e-484e-b87f-0b1a874de567 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:22.052055 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6294e27-9283-4d94-85ba-0b5e66eb5c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:22.052646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6294e27-9283-4d94-85ba-0b5e66eb5c5a admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:22.052932 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6294e27-9283-4d94-85ba-0b5e66eb5c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:22.053717 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6294e27-9283-4d94-85ba-0b5e66eb5c5a admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:22.054408 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6294e27-9283-4d94-85ba-0b5e66eb5c5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:22.056361 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6294e27-9283-4d94-85ba-0b5e66eb5c5a admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:22.076073 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1196/2371] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:25:21 2022] POST /identity/v3/users => generated 292 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:25:22.080041 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f849b-10ab-42fe-8c08-ff19b0e29558 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:22.090227 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f849b-10ab-42fe-8c08-ff19b0e29558 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:22.094837 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b232b075-985b-4b2f-91b1-239d2094f293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:22.095523 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b232b075-985b-4b2f-91b1-239d2094f293 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:22.095814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b232b075-985b-4b2f-91b1-239d2094f293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:22.096095 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b232b075-985b-4b2f-91b1-239d2094f293 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:22.096824 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b232b075-985b-4b2f-91b1-239d2094f293 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:22.098418 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b232b075-985b-4b2f-91b1-239d2094f293 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:22.108653 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1176/2372] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:25:22.112400 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6294e27-9283-4d94-85ba-0b5e66eb5c5a admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:22.122700 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6294e27-9283-4d94-85ba-0b5e66eb5c5a admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:22.127776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a279ca6-40d4-48ac-9975-7bc9d852dfcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:22.128446 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a279ca6-40d4-48ac-9975-7bc9d852dfcd admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:22.128734 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a279ca6-40d4-48ac-9975-7bc9d852dfcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:22.129011 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a279ca6-40d4-48ac-9975-7bc9d852dfcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:22.129758 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a279ca6-40d4-48ac-9975-7bc9d852dfcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:22.132011 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a279ca6-40d4-48ac-9975-7bc9d852dfcd admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:22.143788 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1197/2373] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:25:22.147818 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b232b075-985b-4b2f-91b1-239d2094f293 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:22.157559 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b232b075-985b-4b2f-91b1-239d2094f293 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:22.162137 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e2f558-fed7-4c5c-b526-b4b3902cca65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:22.163004 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2f558-fed7-4c5c-b526-b4b3902cca65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:22.163312 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2f558-fed7-4c5c-b526-b4b3902cca65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:22.163602 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2f558-fed7-4c5c-b526-b4b3902cca65 admin admin] PATH_INFO: `/v3/projects/6ba68b862f0847fa9998f2dbeef9239e/users/fb05d77ba4b34b3698fb523077e6baf3/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:22.177606 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e2f558-fed7-4c5c-b526-b4b3902cca65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ba68b862f0847fa9998f2dbeef9239e, user_id=fb05d77ba4b34b3698fb523077e6baf3, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:22.180789 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e2f558-fed7-4c5c-b526-b4b3902cca65 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:22.193198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1177/2374] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:25:22 2022] PUT /identity/v3/projects/6ba68b862f0847fa9998f2dbeef9239e/users/fb05d77ba4b34b3698fb523077e6baf3/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:25:22.204827 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044bfde8-ef75-445e-902a-3f55286972cd None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:22.205121 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044bfde8-ef75-445e-902a-3f55286972cd None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:22.205402 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044bfde8-ef75-445e-902a-3f55286972cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:22.231719 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-044bfde8-ef75-445e-902a-3f55286972cd None None] MFA Rules not processed for user `fb05d77ba4b34b3698fb523077e6baf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:25:22.232945 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-044bfde8-ef75-445e-902a-3f55286972cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:22.251070 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:25:22.296789 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1198/2375] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:25:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:25:22.301205 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e2f558-fed7-4c5c-b526-b4b3902cca65 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:22.311442 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:INSERT=4,keystone:DELETE=10,keystone:SELECT=86 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:25:22.363063 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e2f558-fed7-4c5c-b526-b4b3902cca65 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:22.367642 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c9a724-eb61-4af0-af1c-5a2dd8dceae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:22.368287 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9a724-eb61-4af0-af1c-5a2dd8dceae6 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:22.368569 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9a724-eb61-4af0-af1c-5a2dd8dceae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:22.368846 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9a724-eb61-4af0-af1c-5a2dd8dceae6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:22.369521 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c9a724-eb61-4af0-af1c-5a2dd8dceae6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:22.371452 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c9a724-eb61-4af0-af1c-5a2dd8dceae6 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:22.396895 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1178/2376] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:25:22 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:25:22.401400 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044bfde8-ef75-445e-902a-3f55286972cd None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:22.411941 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044bfde8-ef75-445e-902a-3f55286972cd None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:22.465505 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec492c9-a599-46b7-a82f-88730c02cd7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:22.466163 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec492c9-a599-46b7-a82f-88730c02cd7d admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:22.466445 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec492c9-a599-46b7-a82f-88730c02cd7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:22.466716 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec492c9-a599-46b7-a82f-88730c02cd7d admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:22.467431 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec492c9-a599-46b7-a82f-88730c02cd7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:22.469336 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec492c9-a599-46b7-a82f-88730c02cd7d admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:22.487976 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1199/2377] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:25:22 2022] POST /identity/v3/users => generated 299 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:25:22.491802 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c9a724-eb61-4af0-af1c-5a2dd8dceae6 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:22.502038 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c9a724-eb61-4af0-af1c-5a2dd8dceae6 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:22.507131 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e830c-a965-4475-ab04-cc440aba0c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:22.507822 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e830c-a965-4475-ab04-cc440aba0c72 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:22.508105 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e830c-a965-4475-ab04-cc440aba0c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:22.508381 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e830c-a965-4475-ab04-cc440aba0c72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:22.509099 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e830c-a965-4475-ab04-cc440aba0c72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:22.510670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e830c-a965-4475-ab04-cc440aba0c72 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:22.521241 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1179/2378] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:25:22.525873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec492c9-a599-46b7-a82f-88730c02cd7d admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:22.535740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec492c9-a599-46b7-a82f-88730c02cd7d admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:22.540326 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d32ba-bce7-4c7d-b783-eb7637bd63a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:22.541199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d32ba-bce7-4c7d-b783-eb7637bd63a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:22.541480 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d32ba-bce7-4c7d-b783-eb7637bd63a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:22.541756 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d32ba-bce7-4c7d-b783-eb7637bd63a8 admin admin] PATH_INFO: `/v3/projects/2755d07846694fa89bfe4ef93a7ff55e/users/ca9c9f6fd22e488aacf4c2e271e36fa7/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:22.556513 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d32ba-bce7-4c7d-b783-eb7637bd63a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2755d07846694fa89bfe4ef93a7ff55e, user_id=ca9c9f6fd22e488aacf4c2e271e36fa7, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:22.559137 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d32ba-bce7-4c7d-b783-eb7637bd63a8 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:22.571648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1200/2379] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:25:22 2022] PUT /identity/v3/projects/2755d07846694fa89bfe4ef93a7ff55e/users/ca9c9f6fd22e488aacf4c2e271e36fa7/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:25:22.575436 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e830c-a965-4475-ab04-cc440aba0c72 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:22.585879 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e830c-a965-4475-ab04-cc440aba0c72 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:22.640670 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2496b5ba-7e96-4a57-afbf-ad3140d0239d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:22.641325 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496b5ba-7e96-4a57-afbf-ad3140d0239d admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:22.641611 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496b5ba-7e96-4a57-afbf-ad3140d0239d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:22.641891 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496b5ba-7e96-4a57-afbf-ad3140d0239d admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:22.642610 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2496b5ba-7e96-4a57-afbf-ad3140d0239d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:22.644495 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2496b5ba-7e96-4a57-afbf-ad3140d0239d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:22.654051 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1180/2380] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:25:22.658319 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d32ba-bce7-4c7d-b783-eb7637bd63a8 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:22.668320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d32ba-bce7-4c7d-b783-eb7637bd63a8 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:22.672827 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1823b1-f201-4c2c-a602-f411f7725e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:22.673693 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1823b1-f201-4c2c-a602-f411f7725e82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:22.673975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1823b1-f201-4c2c-a602-f411f7725e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:22.674249 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1823b1-f201-4c2c-a602-f411f7725e82 admin admin] PATH_INFO: `/v3/projects/2755d07846694fa89bfe4ef93a7ff55e/users/ca9c9f6fd22e488aacf4c2e271e36fa7/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:22.676229 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1823b1-f201-4c2c-a602-f411f7725e82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2755d07846694fa89bfe4ef93a7ff55e, user_id=ca9c9f6fd22e488aacf4c2e271e36fa7, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:22.679337 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1823b1-f201-4c2c-a602-f411f7725e82 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:22.689518 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1201/2381] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:25:22 2022] PUT /identity/v3/projects/2755d07846694fa89bfe4ef93a7ff55e/users/ca9c9f6fd22e488aacf4c2e271e36fa7/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:25:22.700035 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87688a43-e48e-46b9-a3a3-ebec9d473785 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:22.700331 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87688a43-e48e-46b9-a3a3-ebec9d473785 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:22.700615 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87688a43-e48e-46b9-a3a3-ebec9d473785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:22.725799 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-87688a43-e48e-46b9-a3a3-ebec9d473785 None None] MFA Rules not processed for user `ca9c9f6fd22e488aacf4c2e271e36fa7`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:25:22.727004 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-87688a43-e48e-46b9-a3a3-ebec9d473785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:22.794978 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1181/2382] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:25:22 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:25:22.806699 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1823b1-f201-4c2c-a602-f411f7725e82 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:22.808403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-cc1823b1-f201-4c2c-a602-f411f7725e82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:22.862627 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1823b1-f201-4c2c-a602-f411f7725e82 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:22.866630 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7305e070-2db3-42d3-ae7d-f60b40fc87a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:22.867257 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305e070-2db3-42d3-ae7d-f60b40fc87a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:22.867592 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305e070-2db3-42d3-ae7d-f60b40fc87a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:22.867827 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305e070-2db3-42d3-ae7d-f60b40fc87a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:22.895709 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:25:22.926308 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7305e070-2db3-42d3-ae7d-f60b40fc87a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:22.927766 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7305e070-2db3-42d3-ae7d-f60b40fc87a7 service cinder] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:22.936920 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1202/2383] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:25:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:25:27.065772 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87688a43-e48e-46b9-a3a3-ebec9d473785 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:27.067235 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-87688a43-e48e-46b9-a3a3-ebec9d473785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:27.075202 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:25:27.079103 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87688a43-e48e-46b9-a3a3-ebec9d473785 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:27.083630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6a9c7-ec80-41e4-bfaf-46e9aafd25b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:27.084269 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6a9c7-ec80-41e4-bfaf-46e9aafd25b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:27.084558 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6a9c7-ec80-41e4-bfaf-46e9aafd25b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:27.084856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6a9c7-ec80-41e4-bfaf-46e9aafd25b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:27.093889 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd6a9c7-ec80-41e4-bfaf-46e9aafd25b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:27.095344 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd6a9c7-ec80-41e4-bfaf-46e9aafd25b1 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:27.105159 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1182/2384] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:25:27 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:25:31.928636 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7305e070-2db3-42d3-ae7d-f60b40fc87a7 service cinder] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:31.990039 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7305e070-2db3-42d3-ae7d-f60b40fc87a7 service cinder] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:31.994664 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33877065-7b71-43ab-a195-56d432670200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:31.995593 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33877065-7b71-43ab-a195-56d432670200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:31.995875 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33877065-7b71-43ab-a195-56d432670200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:31.996163 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33877065-7b71-43ab-a195-56d432670200 admin admin] PATH_INFO: `/v3/users/fb05d77ba4b34b3698fb523077e6baf3` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:31.997316 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33877065-7b71-43ab-a195-56d432670200 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb05d77ba4b34b3698fb523077e6baf3)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:31.999214 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33877065-7b71-43ab-a195-56d432670200 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:32.044160 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-33877065-7b71-43ab-a195-56d432670200 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb05d77ba4b34b3698fb523077e6baf3'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:32.051741 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-33877065-7b71-43ab-a195-56d432670200 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb05d77ba4b34b3698fb523077e6baf3'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:32.064349 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-33877065-7b71-43ab-a195-56d432670200 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb05d77ba4b34b3698fb523077e6baf3'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:32.071660 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-33877065-7b71-43ab-a195-56d432670200 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb05d77ba4b34b3698fb523077e6baf3'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:32.072322 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-33877065-7b71-43ab-a195-56d432670200 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb05d77ba4b34b3698fb523077e6baf3'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:32.078056 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1203/2385] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:25:31 2022] DELETE /identity/v3/users/fb05d77ba4b34b3698fb523077e6baf3 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:25:32.082393 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0280e9-57d4-4398-8492-ab54b7b05e19 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:32.082834 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0280e9-57d4-4398-8492-ab54b7b05e19 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:32.082938 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0280e9-57d4-4398-8492-ab54b7b05e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:32.100325 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-8d0280e9-57d4-4398-8492-ab54b7b05e19 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:25:32.101725 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8d0280e9-57d4-4398-8492-ab54b7b05e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:32.158365 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1183/2386] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:25:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:25:32.168259 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33877065-7b71-43ab-a195-56d432670200 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:32.169672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-33877065-7b71-43ab-a195-56d432670200 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:32.220837 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:25:32.226777 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33877065-7b71-43ab-a195-56d432670200 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:32.231639 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab8459-f8ca-4208-a92b-eb495c67c99c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:32.232308 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab8459-f8ca-4208-a92b-eb495c67c99c service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:32.232823 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab8459-f8ca-4208-a92b-eb495c67c99c service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:32.233169 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab8459-f8ca-4208-a92b-eb495c67c99c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:32.242627 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab8459-f8ca-4208-a92b-eb495c67c99c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:32.244098 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab8459-f8ca-4208-a92b-eb495c67c99c service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:32.253592 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1204/2387] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:25:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:25:32.515047 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0280e9-57d4-4398-8492-ab54b7b05e19 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:32.516640 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8d0280e9-57d4-4398-8492-ab54b7b05e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:32.530316 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0280e9-57d4-4398-8492-ab54b7b05e19 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:32.534862 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f52c77d-012e-49bb-9729-30bfd85b21df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:32.535755 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f52c77d-012e-49bb-9729-30bfd85b21df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:32.536044 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f52c77d-012e-49bb-9729-30bfd85b21df admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:32.536316 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f52c77d-012e-49bb-9729-30bfd85b21df admin admin] PATH_INFO: `/v3/projects/6ba68b862f0847fa9998f2dbeef9239e` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:32.537489 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f52c77d-012e-49bb-9729-30bfd85b21df admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ba68b862f0847fa9998f2dbeef9239e)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:32.539481 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f52c77d-012e-49bb-9729-30bfd85b21df admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:32.584841 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7f52c77d-012e-49bb-9729-30bfd85b21df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ba68b862f0847fa9998f2dbeef9239e', 'request_id': 'req-7f52c77d-012e-49bb-9729-30bfd85b21df'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:32.590787 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7f52c77d-012e-49bb-9729-30bfd85b21df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ba68b862f0847fa9998f2dbeef9239e', 'request_id': 'req-7f52c77d-012e-49bb-9729-30bfd85b21df'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:32.597969 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7f52c77d-012e-49bb-9729-30bfd85b21df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ba68b862f0847fa9998f2dbeef9239e', 'request_id': 'req-7f52c77d-012e-49bb-9729-30bfd85b21df'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:32.608100 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7f52c77d-012e-49bb-9729-30bfd85b21df admin admin] The token cache is being invalidate because project 6ba68b862f0847fa9998f2dbeef9239e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:25:32.608385 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7f52c77d-012e-49bb-9729-30bfd85b21df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:32.609001 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7f52c77d-012e-49bb-9729-30bfd85b21df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:32.610533 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1184/2388] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:25:32 2022] DELETE /identity/v3/projects/6ba68b862f0847fa9998f2dbeef9239e => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:25:32.614447 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab8459-f8ca-4208-a92b-eb495c67c99c service neutron] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:32.624820 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab8459-f8ca-4208-a92b-eb495c67c99c service neutron] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:32.681502 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3197eb2-3580-462c-a8d8-d17f9973af29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:32.682483 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3197eb2-3580-462c-a8d8-d17f9973af29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:32.682842 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3197eb2-3580-462c-a8d8-d17f9973af29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:32.683194 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3197eb2-3580-462c-a8d8-d17f9973af29 admin admin] PATH_INFO: `/v3/users/ca9c9f6fd22e488aacf4c2e271e36fa7` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:32.684540 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3197eb2-3580-462c-a8d8-d17f9973af29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca9c9f6fd22e488aacf4c2e271e36fa7)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:32.686503 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3197eb2-3580-462c-a8d8-d17f9973af29 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:32.734683 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-a3197eb2-3580-462c-a8d8-d17f9973af29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca9c9f6fd22e488aacf4c2e271e36fa7'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:32.741786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-a3197eb2-3580-462c-a8d8-d17f9973af29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca9c9f6fd22e488aacf4c2e271e36fa7'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:32.753644 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-a3197eb2-3580-462c-a8d8-d17f9973af29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca9c9f6fd22e488aacf4c2e271e36fa7'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:32.760602 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-a3197eb2-3580-462c-a8d8-d17f9973af29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca9c9f6fd22e488aacf4c2e271e36fa7'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:32.761227 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-a3197eb2-3580-462c-a8d8-d17f9973af29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca9c9f6fd22e488aacf4c2e271e36fa7'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:32.767102 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1205/2389] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:25:32 2022] DELETE /identity/v3/users/ca9c9f6fd22e488aacf4c2e271e36fa7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:25:33.030414 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f52c77d-012e-49bb-9729-30bfd85b21df admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:33.031933 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7f52c77d-012e-49bb-9729-30bfd85b21df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:33.051469 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f52c77d-012e-49bb-9729-30bfd85b21df admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:33.109071 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e825f98-f956-40a7-8289-1a7b42939a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:33.110402 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e825f98-f956-40a7-8289-1a7b42939a93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:33.110837 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e825f98-f956-40a7-8289-1a7b42939a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:33.111264 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e825f98-f956-40a7-8289-1a7b42939a93 admin admin] PATH_INFO: `/v3/projects/2755d07846694fa89bfe4ef93a7ff55e` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:33.112561 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e825f98-f956-40a7-8289-1a7b42939a93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2755d07846694fa89bfe4ef93a7ff55e)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:33.114485 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e825f98-f956-40a7-8289-1a7b42939a93 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:33.130903 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:25:33.163619 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6e825f98-f956-40a7-8289-1a7b42939a93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2755d07846694fa89bfe4ef93a7ff55e', 'request_id': 'req-6e825f98-f956-40a7-8289-1a7b42939a93'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:33.169563 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6e825f98-f956-40a7-8289-1a7b42939a93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2755d07846694fa89bfe4ef93a7ff55e', 'request_id': 'req-6e825f98-f956-40a7-8289-1a7b42939a93'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:33.176710 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6e825f98-f956-40a7-8289-1a7b42939a93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2755d07846694fa89bfe4ef93a7ff55e', 'request_id': 'req-6e825f98-f956-40a7-8289-1a7b42939a93'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:33.186923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6e825f98-f956-40a7-8289-1a7b42939a93 admin admin] The token cache is being invalidate because project 2755d07846694fa89bfe4ef93a7ff55e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:25:33.187214 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6e825f98-f956-40a7-8289-1a7b42939a93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:33.187909 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6e825f98-f956-40a7-8289-1a7b42939a93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:33.189485 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1185/2390] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:25:33 2022] DELETE /identity/v3/projects/2755d07846694fa89bfe4ef93a7ff55e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:25:33.195393 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7df6d-4d13-4207-8e7d-73f6b9dddf55 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:33.195876 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7df6d-4d13-4207-8e7d-73f6b9dddf55 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:33.195989 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7df6d-4d13-4207-8e7d-73f6b9dddf55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:33.216368 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-faf7df6d-4d13-4207-8e7d-73f6b9dddf55 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:25:33.217487 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-faf7df6d-4d13-4207-8e7d-73f6b9dddf55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:33.275153 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1206/2391] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:25:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:25:33.283357 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98f28a-7c5f-4b38-99c5-8f57ee6999c8 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:33.283647 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98f28a-7c5f-4b38-99c5-8f57ee6999c8 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:33.283924 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98f28a-7c5f-4b38-99c5-8f57ee6999c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:33.302175 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-6d98f28a-7c5f-4b38-99c5-8f57ee6999c8 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:25:33.303400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6d98f28a-7c5f-4b38-99c5-8f57ee6999c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:33.311111 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1186/2392] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:25:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:25:33.315027 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7df6d-4d13-4207-8e7d-73f6b9dddf55 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:33.324355 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7df6d-4d13-4207-8e7d-73f6b9dddf55 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:33.328861 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162626aa-edc1-42e7-92f3-f70cec500ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:33.329505 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162626aa-edc1-42e7-92f3-f70cec500ef7 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:33.329785 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162626aa-edc1-42e7-92f3-f70cec500ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:33.330059 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162626aa-edc1-42e7-92f3-f70cec500ef7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:33.330771 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162626aa-edc1-42e7-92f3-f70cec500ef7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:25:33.331075 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162626aa-edc1-42e7-92f3-f70cec500ef7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:33.332700 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162626aa-edc1-42e7-92f3-f70cec500ef7 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:33.335463 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:25:33.349090 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1207/2393] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:25:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:25:33.352952 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d98f28a-7c5f-4b38-99c5-8f57ee6999c8 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:33.362291 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d98f28a-7c5f-4b38-99c5-8f57ee6999c8 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:33.366967 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae48ce-0737-4d30-803c-2fe260f0608e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:33.367785 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae48ce-0737-4d30-803c-2fe260f0608e admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:33.368069 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae48ce-0737-4d30-803c-2fe260f0608e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:33.368347 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae48ce-0737-4d30-803c-2fe260f0608e admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:33.369053 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfae48ce-0737-4d30-803c-2fe260f0608e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:33.370953 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfae48ce-0737-4d30-803c-2fe260f0608e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:33.395713 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1187/2394] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:25:33 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:25:33.400308 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162626aa-edc1-42e7-92f3-f70cec500ef7 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:33.410142 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162626aa-edc1-42e7-92f3-f70cec500ef7 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:33.464451 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aef510-64a2-4d39-a44d-d8088e113980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:33.465118 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aef510-64a2-4d39-a44d-d8088e113980 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:33.465405 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aef510-64a2-4d39-a44d-d8088e113980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:33.465686 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aef510-64a2-4d39-a44d-d8088e113980 admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:33.466440 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aef510-64a2-4d39-a44d-d8088e113980 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:33.468333 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aef510-64a2-4d39-a44d-d8088e113980 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:33.488765 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1208/2395] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:25:33 2022] POST /identity/v3/users => generated 287 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:25:33.492853 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae48ce-0737-4d30-803c-2fe260f0608e admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:33.503014 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae48ce-0737-4d30-803c-2fe260f0608e admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:33.508110 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00221a9f-a637-4736-93fb-ee07d4cfbbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:33.508763 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00221a9f-a637-4736-93fb-ee07d4cfbbee admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:33.509049 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00221a9f-a637-4736-93fb-ee07d4cfbbee admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:33.509327 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00221a9f-a637-4736-93fb-ee07d4cfbbee admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:33.510055 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00221a9f-a637-4736-93fb-ee07d4cfbbee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:33.511683 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00221a9f-a637-4736-93fb-ee07d4cfbbee admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:33.522132 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1188/2396] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:25:33.525988 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aef510-64a2-4d39-a44d-d8088e113980 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:33.535669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aef510-64a2-4d39-a44d-d8088e113980 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:33.540624 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798b4e0-42b0-4595-a5df-2a4e131ae9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:33.541269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798b4e0-42b0-4595-a5df-2a4e131ae9b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:33.541548 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798b4e0-42b0-4595-a5df-2a4e131ae9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:33.541819 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798b4e0-42b0-4595-a5df-2a4e131ae9b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:33.542527 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798b4e0-42b0-4595-a5df-2a4e131ae9b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:33.544142 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798b4e0-42b0-4595-a5df-2a4e131ae9b7 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:33.554424 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1209/2397] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:25:33.558422 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00221a9f-a637-4736-93fb-ee07d4cfbbee admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:33.568170 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00221a9f-a637-4736-93fb-ee07d4cfbbee admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:33.572779 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9560a09c-2644-4384-b4b0-12bccd19f13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:33.573647 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560a09c-2644-4384-b4b0-12bccd19f13b admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:33.573932 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560a09c-2644-4384-b4b0-12bccd19f13b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:33.574213 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560a09c-2644-4384-b4b0-12bccd19f13b admin admin] PATH_INFO: `/v3/projects/72012398cf024f3ebda8769a6c436553/users/3158a9a73a574f2b9820015289f0e840/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:33.588727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9560a09c-2644-4384-b4b0-12bccd19f13b admin admin] RBAC: Authorizing `identity:create_grant(project_id=72012398cf024f3ebda8769a6c436553, user_id=3158a9a73a574f2b9820015289f0e840, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:33.591397 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9560a09c-2644-4384-b4b0-12bccd19f13b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:33.604472 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1189/2398] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:25:33 2022] PUT /identity/v3/projects/72012398cf024f3ebda8769a6c436553/users/3158a9a73a574f2b9820015289f0e840/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:25:33.668125 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948dfcb-5041-4156-bff4-bcf3ced9eb18 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:33.668419 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948dfcb-5041-4156-bff4-bcf3ced9eb18 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:33.668703 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948dfcb-5041-4156-bff4-bcf3ced9eb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:33.694936 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-5948dfcb-5041-4156-bff4-bcf3ced9eb18 None None] MFA Rules not processed for user `3158a9a73a574f2b9820015289f0e840`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:25:33.696185 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-5948dfcb-5041-4156-bff4-bcf3ced9eb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:33.762909 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1210/2399] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:25:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:25:33.774101 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9560a09c-2644-4384-b4b0-12bccd19f13b admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:33.775835 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9560a09c-2644-4384-b4b0-12bccd19f13b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:33.830373 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9560a09c-2644-4384-b4b0-12bccd19f13b admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:33.833936 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754c2453-ec5e-4250-a56c-a4f835fd7059 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:33.834564 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c2453-ec5e-4250-a56c-a4f835fd7059 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:33.835393 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c2453-ec5e-4250-a56c-a4f835fd7059 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:33.835673 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c2453-ec5e-4250-a56c-a4f835fd7059 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:33.843957 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754c2453-ec5e-4250-a56c-a4f835fd7059 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:33.845412 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754c2453-ec5e-4250-a56c-a4f835fd7059 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:33.854225 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1190/2400] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:25:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:25:33.873988 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5948dfcb-5041-4156-bff4-bcf3ced9eb18 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:33.933753 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5948dfcb-5041-4156-bff4-bcf3ced9eb18 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:33.938333 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e8a9d-9de0-497f-a6d7-1a6b61fad0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:33.939239 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e8a9d-9de0-497f-a6d7-1a6b61fad0cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:33.939550 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e8a9d-9de0-497f-a6d7-1a6b61fad0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:33.939827 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e8a9d-9de0-497f-a6d7-1a6b61fad0cd admin admin] PATH_INFO: `/v3/users/3158a9a73a574f2b9820015289f0e840` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:33.941002 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e8a9d-9de0-497f-a6d7-1a6b61fad0cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3158a9a73a574f2b9820015289f0e840)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:33.942887 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e8a9d-9de0-497f-a6d7-1a6b61fad0cd admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:33.947786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:25:33.990117 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-274e8a9d-9de0-497f-a6d7-1a6b61fad0cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3158a9a73a574f2b9820015289f0e840'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:33.996618 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-274e8a9d-9de0-497f-a6d7-1a6b61fad0cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3158a9a73a574f2b9820015289f0e840'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:34.008421 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-274e8a9d-9de0-497f-a6d7-1a6b61fad0cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3158a9a73a574f2b9820015289f0e840'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:34.015040 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-274e8a9d-9de0-497f-a6d7-1a6b61fad0cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3158a9a73a574f2b9820015289f0e840'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:34.015668 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-274e8a9d-9de0-497f-a6d7-1a6b61fad0cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3158a9a73a574f2b9820015289f0e840'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:34.021801 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1211/2401] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:25:33 2022] DELETE /identity/v3/users/3158a9a73a574f2b9820015289f0e840 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:25:34.026790 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143e958-c2d3-45c6-a90a-4aac2b4e21bf None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:34.027101 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143e958-c2d3-45c6-a90a-4aac2b4e21bf None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:34.027419 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143e958-c2d3-45c6-a90a-4aac2b4e21bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:34.044424 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-a143e958-c2d3-45c6-a90a-4aac2b4e21bf None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:25:34.045612 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a143e958-c2d3-45c6-a90a-4aac2b4e21bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:34.084078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:25:34.105664 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1191/2402] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:25:34.115321 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e8a9d-9de0-497f-a6d7-1a6b61fad0cd admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:34.117240 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-274e8a9d-9de0-497f-a6d7-1a6b61fad0cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:34.173194 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e8a9d-9de0-497f-a6d7-1a6b61fad0cd admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:34.176899 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c49225-e9be-4695-83ec-0c2432e0e7a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:34.177562 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c49225-e9be-4695-83ec-0c2432e0e7a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:34.177843 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c49225-e9be-4695-83ec-0c2432e0e7a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:34.178119 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c49225-e9be-4695-83ec-0c2432e0e7a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:34.186995 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c49225-e9be-4695-83ec-0c2432e0e7a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:34.188433 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c49225-e9be-4695-83ec-0c2432e0e7a6 service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:34.197768 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1212/2403] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:25:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:25:34.458522 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a143e958-c2d3-45c6-a90a-4aac2b4e21bf None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:34.460040 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a143e958-c2d3-45c6-a90a-4aac2b4e21bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:34.474642 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a143e958-c2d3-45c6-a90a-4aac2b4e21bf None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:34.480130 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d133d5c-23fb-4dd0-952b-933c2554b75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:34.481139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d133d5c-23fb-4dd0-952b-933c2554b75c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:34.481425 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d133d5c-23fb-4dd0-952b-933c2554b75c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:34.481708 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d133d5c-23fb-4dd0-952b-933c2554b75c admin admin] PATH_INFO: `/v3/projects/72012398cf024f3ebda8769a6c436553` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:34.483024 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d133d5c-23fb-4dd0-952b-933c2554b75c admin admin] RBAC: Authorizing `identity:delete_project(project_id=72012398cf024f3ebda8769a6c436553)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:34.485005 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d133d5c-23fb-4dd0-952b-933c2554b75c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:34.530440 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7d133d5c-23fb-4dd0-952b-933c2554b75c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72012398cf024f3ebda8769a6c436553', 'request_id': 'req-7d133d5c-23fb-4dd0-952b-933c2554b75c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:34.536310 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7d133d5c-23fb-4dd0-952b-933c2554b75c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72012398cf024f3ebda8769a6c436553', 'request_id': 'req-7d133d5c-23fb-4dd0-952b-933c2554b75c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:34.543827 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7d133d5c-23fb-4dd0-952b-933c2554b75c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72012398cf024f3ebda8769a6c436553', 'request_id': 'req-7d133d5c-23fb-4dd0-952b-933c2554b75c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:34.555111 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7d133d5c-23fb-4dd0-952b-933c2554b75c admin admin] The token cache is being invalidate because project 72012398cf024f3ebda8769a6c436553 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:25:34.555431 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7d133d5c-23fb-4dd0-952b-933c2554b75c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:34.556056 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7d133d5c-23fb-4dd0-952b-933c2554b75c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:25:34.557613 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1192/2404] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:25:34 2022] DELETE /identity/v3/projects/72012398cf024f3ebda8769a6c436553 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:25:34.562633 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659911bc-d08a-4969-9d07-839e64b4f503 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:34.562924 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659911bc-d08a-4969-9d07-839e64b4f503 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:34.563200 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659911bc-d08a-4969-9d07-839e64b4f503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:34.580252 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-659911bc-d08a-4969-9d07-839e64b4f503 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:25:34.581437 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-659911bc-d08a-4969-9d07-839e64b4f503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:34.638952 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1213/2405] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:25:34.646913 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc0783-41b8-4802-8397-3d3218e500a8 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:34.647209 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc0783-41b8-4802-8397-3d3218e500a8 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:34.648011 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc0783-41b8-4802-8397-3d3218e500a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:34.666245 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-e9dc0783-41b8-4802-8397-3d3218e500a8 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:25:34.667473 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e9dc0783-41b8-4802-8397-3d3218e500a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:34.675167 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1193/2406] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:25:34.679249 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659911bc-d08a-4969-9d07-839e64b4f503 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:34.688697 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659911bc-d08a-4969-9d07-839e64b4f503 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:34.693220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd619f01-9711-4e85-ae3d-f6772bf6fbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:34.693859 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd619f01-9711-4e85-ae3d-f6772bf6fbbd admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:34.694142 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd619f01-9711-4e85-ae3d-f6772bf6fbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:34.694417 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd619f01-9711-4e85-ae3d-f6772bf6fbbd admin admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:34.695132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd619f01-9711-4e85-ae3d-f6772bf6fbbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:25:34.695469 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd619f01-9711-4e85-ae3d-f6772bf6fbbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:34.697041 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd619f01-9711-4e85-ae3d-f6772bf6fbbd admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:34.708172 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1214/2407] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:25:34.712799 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc0783-41b8-4802-8397-3d3218e500a8 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:34.721685 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc0783-41b8-4802-8397-3d3218e500a8 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:34.726836 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e872331b-4430-4979-96ca-900ccf603a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:34.727508 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872331b-4430-4979-96ca-900ccf603a0f admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:34.727795 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872331b-4430-4979-96ca-900ccf603a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:34.728069 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872331b-4430-4979-96ca-900ccf603a0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:34.728747 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e872331b-4430-4979-96ca-900ccf603a0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:34.730644 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e872331b-4430-4979-96ca-900ccf603a0f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:34.754915 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1194/2408] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:25:34 2022] POST /identity/v3/projects => generated 391 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:25:34.758759 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd619f01-9711-4e85-ae3d-f6772bf6fbbd admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:34.768662 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd619f01-9711-4e85-ae3d-f6772bf6fbbd admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:34.778095 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:25:34.827014 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca30d774-4de2-4909-8b3d-235fefc17613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:34.827716 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca30d774-4de2-4909-8b3d-235fefc17613 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:34.828001 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca30d774-4de2-4909-8b3d-235fefc17613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:34.828285 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca30d774-4de2-4909-8b3d-235fefc17613 admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:34.828974 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca30d774-4de2-4909-8b3d-235fefc17613 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:34.830861 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca30d774-4de2-4909-8b3d-235fefc17613 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:34.851164 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1215/2409] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:25:34 2022] POST /identity/v3/users => generated 296 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:25:34.855586 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e872331b-4430-4979-96ca-900ccf603a0f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:34.865742 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e872331b-4430-4979-96ca-900ccf603a0f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:34.870837 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bee6e6-eb7e-4041-8d59-34fbc0692b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:34.871543 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bee6e6-eb7e-4041-8d59-34fbc0692b8d admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:34.871818 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bee6e6-eb7e-4041-8d59-34fbc0692b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:34.872099 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bee6e6-eb7e-4041-8d59-34fbc0692b8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:34.872829 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bee6e6-eb7e-4041-8d59-34fbc0692b8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:34.874534 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bee6e6-eb7e-4041-8d59-34fbc0692b8d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:34.886106 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1195/2410] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:25:34.890063 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca30d774-4de2-4909-8b3d-235fefc17613 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:34.900160 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca30d774-4de2-4909-8b3d-235fefc17613 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:34.904656 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14adc47-855b-4081-8f7e-d887e2909e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:34.905309 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14adc47-855b-4081-8f7e-d887e2909e52 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:34.905589 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14adc47-855b-4081-8f7e-d887e2909e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:34.905868 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14adc47-855b-4081-8f7e-d887e2909e52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:34.906582 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14adc47-855b-4081-8f7e-d887e2909e52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:34.908200 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14adc47-855b-4081-8f7e-d887e2909e52 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:34.917832 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1216/2411] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:25:34.921681 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bee6e6-eb7e-4041-8d59-34fbc0692b8d admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:34.932633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bee6e6-eb7e-4041-8d59-34fbc0692b8d admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:34.937219 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1faf86d-51f6-46b8-b434-c9a987646e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:34.938097 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1faf86d-51f6-46b8-b434-c9a987646e42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:34.938384 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1faf86d-51f6-46b8-b434-c9a987646e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:34.938666 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1faf86d-51f6-46b8-b434-c9a987646e42 admin admin] PATH_INFO: `/v3/projects/3c8d7e08e969475fb341717632d051ac/users/758547c3625345dc965c7030e61678a0/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:34.953469 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1faf86d-51f6-46b8-b434-c9a987646e42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c8d7e08e969475fb341717632d051ac, user_id=758547c3625345dc965c7030e61678a0, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:34.956157 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1faf86d-51f6-46b8-b434-c9a987646e42 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:34.970114 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1196/2412] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:25:34 2022] PUT /identity/v3/projects/3c8d7e08e969475fb341717632d051ac/users/758547c3625345dc965c7030e61678a0/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:25:34.980769 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0543ed85-3ebf-493a-bddc-13bb4bd8ecbe None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:34.981060 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0543ed85-3ebf-493a-bddc-13bb4bd8ecbe None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:34.981339 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0543ed85-3ebf-493a-bddc-13bb4bd8ecbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:35.007351 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-0543ed85-3ebf-493a-bddc-13bb4bd8ecbe None None] MFA Rules not processed for user `758547c3625345dc965c7030e61678a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:25:35.008583 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-0543ed85-3ebf-493a-bddc-13bb4bd8ecbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:35.070267 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1217/2413] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:25:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:25:35.080736 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1faf86d-51f6-46b8-b434-c9a987646e42 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:35.082451 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c1faf86d-51f6-46b8-b434-c9a987646e42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:35.138456 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1faf86d-51f6-46b8-b434-c9a987646e42 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:35.142116 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e1b60-44df-4722-960f-c61b825938af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:35.142753 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e1b60-44df-4722-960f-c61b825938af service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:35.143036 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e1b60-44df-4722-960f-c61b825938af service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:35.143336 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e1b60-44df-4722-960f-c61b825938af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:35.153084 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39e1b60-44df-4722-960f-c61b825938af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:35.154514 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39e1b60-44df-4722-960f-c61b825938af service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:35.163309 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1197/2414] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:25:35 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:25:45.066836 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=60,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:25:45.159257 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:25:48.837477 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0543ed85-3ebf-493a-bddc-13bb4bd8ecbe None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:48.838893 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-0543ed85-3ebf-493a-bddc-13bb4bd8ecbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:48.897525 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0543ed85-3ebf-493a-bddc-13bb4bd8ecbe None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:48.901333 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc06e07-bc49-4e57-9d53-b8db591c407c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbaf66d6e1f54047917cc615546bbd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:48.901976 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc06e07-bc49-4e57-9d53-b8db591c407c service glance] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:48.902254 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc06e07-bc49-4e57-9d53-b8db591c407c service glance] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:48.902525 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc06e07-bc49-4e57-9d53-b8db591c407c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:48.904276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-3bc06e07-bc49-4e57-9d53-b8db591c407c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:48.971973 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc06e07-bc49-4e57-9d53-b8db591c407c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:48.973395 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc06e07-bc49-4e57-9d53-b8db591c407c service glance] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:48.984197 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1218/2415] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:25:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:25:49.225810 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e1b60-44df-4722-960f-c61b825938af service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:49.227903 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c39e1b60-44df-4722-960f-c61b825938af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:49.283662 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e1b60-44df-4722-960f-c61b825938af service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:49.287183 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe27cf-adff-4bc6-9c71-95580dd88edc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:49.287851 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe27cf-adff-4bc6-9c71-95580dd88edc service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:49.288142 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe27cf-adff-4bc6-9c71-95580dd88edc service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:49.288398 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe27cf-adff-4bc6-9c71-95580dd88edc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:49.290180 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-dbbe27cf-adff-4bc6-9c71-95580dd88edc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:49.317658 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbe27cf-adff-4bc6-9c71-95580dd88edc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:49.319146 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbe27cf-adff-4bc6-9c71-95580dd88edc service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:49.332098 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1198/2416] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:25:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:25:51.624140 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc06e07-bc49-4e57-9d53-b8db591c407c service glance] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:51.625536 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-3bc06e07-bc49-4e57-9d53-b8db591c407c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:51.639013 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc06e07-bc49-4e57-9d53-b8db591c407c service glance] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:51.644474 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae20dfd-6bc4-498d-85e8-355f4d806df0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:51.645381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae20dfd-6bc4-498d-85e8-355f4d806df0 service nova] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:51.645653 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae20dfd-6bc4-498d-85e8-355f4d806df0 service nova] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:51.645932 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae20dfd-6bc4-498d-85e8-355f4d806df0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:51.647686 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-0ae20dfd-6bc4-498d-85e8-355f4d806df0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:51.724303 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae20dfd-6bc4-498d-85e8-355f4d806df0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:51.725890 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae20dfd-6bc4-498d-85e8-355f4d806df0 service nova] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:51.746472 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1219/2417] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:25:51 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:25:52.006044 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe27cf-adff-4bc6-9c71-95580dd88edc service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:25:52.007482 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-dbbe27cf-adff-4bc6-9c71-95580dd88edc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:52.021238 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe27cf-adff-4bc6-9c71-95580dd88edc service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:25:52.027159 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd03c2-8ddc-41f6-9da3-6328f9bc79a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:25:52.027819 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd03c2-8ddc-41f6-9da3-6328f9bc79a1 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:25:52.028106 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd03c2-8ddc-41f6-9da3-6328f9bc79a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:25:52.028382 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd03c2-8ddc-41f6-9da3-6328f9bc79a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:25:52.030083 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9bcd03c2-8ddc-41f6-9da3-6328f9bc79a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:25:52.109159 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd03c2-8ddc-41f6-9da3-6328f9bc79a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:25:52.111425 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd03c2-8ddc-41f6-9da3-6328f9bc79a1 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:25:52.130170 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1199/2418] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:25:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:26:01.742222 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=95 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:26:02.127629 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=67 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:26:03.486396 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a068d4e1-8694-4123-80c5-877e9ad642cf None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:26:03.486886 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a068d4e1-8694-4123-80c5-877e9ad642cf None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:26:03.487007 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a068d4e1-8694-4123-80c5-877e9ad642cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:26:03.512632 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-a068d4e1-8694-4123-80c5-877e9ad642cf None None] MFA Rules not processed for user `ef7c1e449b864b50b3c71c39f19968a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:26:03.513822 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-a068d4e1-8694-4123-80c5-877e9ad642cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:26:03.520776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1220/2419] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:26:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:26:03.816682 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd03c2-8ddc-41f6-9da3-6328f9bc79a1 service nova] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:26:03.826580 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd03c2-8ddc-41f6-9da3-6328f9bc79a1 service nova] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:26:03.830243 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292fc45-7a91-4fd8-b357-ce08f324a49f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:26:03.830965 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292fc45-7a91-4fd8-b357-ce08f324a49f service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:26:03.831354 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292fc45-7a91-4fd8-b357-ce08f324a49f service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:26:03.831729 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292fc45-7a91-4fd8-b357-ce08f324a49f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:26:03.833596 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f292fc45-7a91-4fd8-b357-ce08f324a49f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:26:03.843753 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f292fc45-7a91-4fd8-b357-ce08f324a49f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:26:03.845264 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f292fc45-7a91-4fd8-b357-ce08f324a49f service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:26:03.855060 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1200/2420] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:26:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:26:04.182247 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a068d4e1-8694-4123-80c5-877e9ad642cf None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:26:04.183770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-a068d4e1-8694-4123-80c5-877e9ad642cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:26:04.263969 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a068d4e1-8694-4123-80c5-877e9ad642cf None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:26:04.269015 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d74d4-15f4-4608-a571-55a48f4a0a8f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be901786f8934d559e0bac5d622f97cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:26:04.269671 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d74d4-15f4-4608-a571-55a48f4a0a8f service placement] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:26:04.269991 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d74d4-15f4-4608-a571-55a48f4a0a8f service placement] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:26:04.270523 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d74d4-15f4-4608-a571-55a48f4a0a8f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:26:04.272323 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-700d74d4-15f4-4608-a571-55a48f4a0a8f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:26:04.283555 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d74d4-15f4-4608-a571-55a48f4a0a8f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:26:04.285123 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d74d4-15f4-4608-a571-55a48f4a0a8f service placement] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:26:04.295403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1221/2421] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:26:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:26:08.127444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292fc45-7a91-4fd8-b357-ce08f324a49f service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:26:08.165257 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f91de-4db0-4f71-b3c2-8ca46e7f9a27 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:26:08.165811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f91de-4db0-4f71-b3c2-8ca46e7f9a27 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:26:08.165811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f91de-4db0-4f71-b3c2-8ca46e7f9a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:26:08.184076 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-248f91de-4db0-4f71-b3c2-8ca46e7f9a27 None None] MFA Rules not processed for user `331c97a12f1b4c2eb1eefa61056ce089`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:26:08.185299 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-248f91de-4db0-4f71-b3c2-8ca46e7f9a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:26:08.193474 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292fc45-7a91-4fd8-b357-ce08f324a49f service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:26:08.198471 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c4ff7c-037f-469b-a418-3fdb101c53fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:26:08.199362 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4ff7c-037f-469b-a418-3fdb101c53fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:26:08.199653 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4ff7c-037f-469b-a418-3fdb101c53fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:26:08.199936 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4ff7c-037f-469b-a418-3fdb101c53fb admin admin] PATH_INFO: `/v3/users/758547c3625345dc965c7030e61678a0` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:26:08.201129 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c4ff7c-037f-469b-a418-3fdb101c53fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=758547c3625345dc965c7030e61678a0)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:26:08.203129 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c4ff7c-037f-469b-a418-3fdb101c53fb admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:26:08.247733 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1222/2422] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:26:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:26:08.258030 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-49c4ff7c-037f-469b-a418-3fdb101c53fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '758547c3625345dc965c7030e61678a0'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:26:08.265381 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-49c4ff7c-037f-469b-a418-3fdb101c53fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '758547c3625345dc965c7030e61678a0'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:26:08.278590 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-49c4ff7c-037f-469b-a418-3fdb101c53fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '758547c3625345dc965c7030e61678a0'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:26:08.279264 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-49c4ff7c-037f-469b-a418-3fdb101c53fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '758547c3625345dc965c7030e61678a0'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:26:08.287451 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-49c4ff7c-037f-469b-a418-3fdb101c53fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '758547c3625345dc965c7030e61678a0'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:26:08.293968 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1201/2423] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:26:08 2022] DELETE /identity/v3/users/758547c3625345dc965c7030e61678a0 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:26:08.298516 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40114f-af88-494a-9932-9f351077aad2 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:26:08.298817 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40114f-af88-494a-9932-9f351077aad2 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:26:08.299099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40114f-af88-494a-9932-9f351077aad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:26:08.317195 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-fe40114f-af88-494a-9932-9f351077aad2 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:26:08.318394 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-fe40114f-af88-494a-9932-9f351077aad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:26:08.378848 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1223/2424] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:26:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:26:08.388520 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c4ff7c-037f-469b-a418-3fdb101c53fb admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:26:08.389978 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-49c4ff7c-037f-469b-a418-3fdb101c53fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:26:08.447846 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c4ff7c-037f-469b-a418-3fdb101c53fb admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:26:08.451524 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce8dca4-8721-4067-b1d9-c21f2ed81d06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:26:08.452172 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8dca4-8721-4067-b1d9-c21f2ed81d06 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:26:08.452463 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8dca4-8721-4067-b1d9-c21f2ed81d06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:26:08.452752 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8dca4-8721-4067-b1d9-c21f2ed81d06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:26:08.462432 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce8dca4-8721-4067-b1d9-c21f2ed81d06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:26:08.463915 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce8dca4-8721-4067-b1d9-c21f2ed81d06 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:26:08.473819 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1202/2425] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:26:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 16:26:08.879583 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40114f-af88-494a-9932-9f351077aad2 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:26:08.881017 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-fe40114f-af88-494a-9932-9f351077aad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:26:08.894010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40114f-af88-494a-9932-9f351077aad2 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:26:08.900968 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f13f20-a787-4542-afbf-a574b38e39ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:26:08.902095 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f13f20-a787-4542-afbf-a574b38e39ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:26:08.902606 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f13f20-a787-4542-afbf-a574b38e39ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:26:08.903225 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f13f20-a787-4542-afbf-a574b38e39ea admin admin] PATH_INFO: `/v3/projects/3c8d7e08e969475fb341717632d051ac` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:26:08.916186 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f13f20-a787-4542-afbf-a574b38e39ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c8d7e08e969475fb341717632d051ac)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:26:08.919000 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f13f20-a787-4542-afbf-a574b38e39ea admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:26:08.942362 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:26:08.976026 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-c3f13f20-a787-4542-afbf-a574b38e39ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c8d7e08e969475fb341717632d051ac', 'request_id': 'req-c3f13f20-a787-4542-afbf-a574b38e39ea'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:26:08.982632 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-c3f13f20-a787-4542-afbf-a574b38e39ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c8d7e08e969475fb341717632d051ac', 'request_id': 'req-c3f13f20-a787-4542-afbf-a574b38e39ea'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:26:08.990871 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-c3f13f20-a787-4542-afbf-a574b38e39ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c8d7e08e969475fb341717632d051ac', 'request_id': 'req-c3f13f20-a787-4542-afbf-a574b38e39ea'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:26:09.001873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-c3f13f20-a787-4542-afbf-a574b38e39ea admin admin] The token cache is being invalidate because project 3c8d7e08e969475fb341717632d051ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:26:09.002164 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-c3f13f20-a787-4542-afbf-a574b38e39ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:26:09.002855 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-c3f13f20-a787-4542-afbf-a574b38e39ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:26:09.004646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1224/2426] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:26:08 2022] DELETE /identity/v3/projects/3c8d7e08e969475fb341717632d051ac => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:26:15.779558 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce8dca4-8721-4067-b1d9-c21f2ed81d06 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:26:15.790605 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce8dca4-8721-4067-b1d9-c21f2ed81d06 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:26:15.795434 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:26:15.843785 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f23cd-37b7-4f21-8e4d-3332e2e54ac5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:26:15.844442 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f23cd-37b7-4f21-8e4d-3332e2e54ac5 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:26:15.844735 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f23cd-37b7-4f21-8e4d-3332e2e54ac5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:26:15.845014 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f23cd-37b7-4f21-8e4d-3332e2e54ac5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:26:15.846848 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5a2f23cd-37b7-4f21-8e4d-3332e2e54ac5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:26:15.857519 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f23cd-37b7-4f21-8e4d-3332e2e54ac5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:26:15.858920 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f23cd-37b7-4f21-8e4d-3332e2e54ac5 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:26:15.868439 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1203/2427] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:26:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:26:17.873397 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52768d98-ea9a-4e22-a351-c3e16677ee5d None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:26:17.873905 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52768d98-ea9a-4e22-a351-c3e16677ee5d None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:26:17.874035 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52768d98-ea9a-4e22-a351-c3e16677ee5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:26:17.892201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-52768d98-ea9a-4e22-a351-c3e16677ee5d None None] MFA Rules not processed for user `875140cf51d9410da45f8c3805b26204`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:26:17.893395 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-52768d98-ea9a-4e22-a351-c3e16677ee5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:26:17.948448 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1225/2428] 10.222.0.41 () {58 vars in 1009 bytes} [Fri Jun 3 16:26:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:26:17.988579 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f23cd-37b7-4f21-8e4d-3332e2e54ac5 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:26:17.990087 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5a2f23cd-37b7-4f21-8e4d-3332e2e54ac5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:26:18.059188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f23cd-37b7-4f21-8e4d-3332e2e54ac5 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:26:18.064339 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8f16ad-4505-40b3-93ae-587be180307f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5f513b781a4a73b443cded13359eda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:26:18.064984 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f16ad-4505-40b3-93ae-587be180307f service swift] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:26:18.065272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f16ad-4505-40b3-93ae-587be180307f service swift] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:26:18.065556 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f16ad-4505-40b3-93ae-587be180307f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:26:18.073793 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8f16ad-4505-40b3-93ae-587be180307f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:26:18.075458 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8f16ad-4505-40b3-93ae-587be180307f service swift] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:26:18.091495 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1204/2429] 10.222.0.41 () {60 vars in 1384 bytes} [Fri Jun 3 16:26:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 16:26:27.941376 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:26:28.082499 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=54 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:26:29.748483 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52768d98-ea9a-4e22-a351-c3e16677ee5d None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:26:29.750014 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-52768d98-ea9a-4e22-a351-c3e16677ee5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:26:29.858983 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52768d98-ea9a-4e22-a351-c3e16677ee5d None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:26:29.864485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f19e7-485b-4aff-92b9-25c9a456780a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:26:29.867907 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f19e7-485b-4aff-92b9-25c9a456780a service nova] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:26:29.868238 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f19e7-485b-4aff-92b9-25c9a456780a service nova] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:26:29.868527 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f19e7-485b-4aff-92b9-25c9a456780a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:26:29.870362 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-c94f19e7-485b-4aff-92b9-25c9a456780a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:26:29.885623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94f19e7-485b-4aff-92b9-25c9a456780a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:26:29.887771 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94f19e7-485b-4aff-92b9-25c9a456780a service nova] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:26:29.901374 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1226/2430] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:26:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:26:30.844639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2b751-0078-49ac-a9eb-91333227d05f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:26:30.845170 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2b751-0078-49ac-a9eb-91333227d05f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:26:30.845170 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2b751-0078-49ac-a9eb-91333227d05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:26:30.863049 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-84a2b751-0078-49ac-a9eb-91333227d05f None None] MFA Rules not processed for user `875140cf51d9410da45f8c3805b26204`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:26:30.864307 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-84a2b751-0078-49ac-a9eb-91333227d05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:26:30.871722 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1205/2431] 10.222.0.41 () {58 vars in 967 bytes} [Fri Jun 3 16:26:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:26:30.883630 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f19e7-485b-4aff-92b9-25c9a456780a service nova] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:26:30.894847 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f19e7-485b-4aff-92b9-25c9a456780a service nova] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:26:30.900652 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d000d2a3-0bed-4609-b889-9f68f994fdf4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5f513b781a4a73b443cded13359eda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:26:30.901368 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d000d2a3-0bed-4609-b889-9f68f994fdf4 service swift] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:26:30.901721 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d000d2a3-0bed-4609-b889-9f68f994fdf4 service swift] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:26:30.902214 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d000d2a3-0bed-4609-b889-9f68f994fdf4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:26:30.911352 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d000d2a3-0bed-4609-b889-9f68f994fdf4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:26:30.912930 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d000d2a3-0bed-4609-b889-9f68f994fdf4 service swift] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:26:30.923205 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1227/2432] 10.222.0.41 () {60 vars in 1384 bytes} [Fri Jun 3 16:26:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 03 16:26:31.149052 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a2b751-0078-49ac-a9eb-91333227d05f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:26:31.151231 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-84a2b751-0078-49ac-a9eb-91333227d05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:26:31.165039 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a2b751-0078-49ac-a9eb-91333227d05f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:26:31.168779 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e355b-e1b6-40c7-93d6-da4f7f79201c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:26:31.169458 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e355b-e1b6-40c7-93d6-da4f7f79201c service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:26:31.169744 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e355b-e1b6-40c7-93d6-da4f7f79201c service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:26:31.170025 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e355b-e1b6-40c7-93d6-da4f7f79201c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:26:31.171854 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-217e355b-e1b6-40c7-93d6-da4f7f79201c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:26:31.236566 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217e355b-e1b6-40c7-93d6-da4f7f79201c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:26:31.237978 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217e355b-e1b6-40c7-93d6-da4f7f79201c service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:26:31.248378 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1206/2433] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:26:31 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:26:35.463042 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d000d2a3-0bed-4609-b889-9f68f994fdf4 service swift] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:26:35.464677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-d000d2a3-0bed-4609-b889-9f68f994fdf4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:26:35.578604 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d000d2a3-0bed-4609-b889-9f68f994fdf4 service swift] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:26:35.583938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ae60d2-4ab0-42a1-ad1e-f5e37c171f23 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be901786f8934d559e0bac5d622f97cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:26:35.584665 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ae60d2-4ab0-42a1-ad1e-f5e37c171f23 service placement] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:26:35.585164 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ae60d2-4ab0-42a1-ad1e-f5e37c171f23 service placement] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:26:35.585513 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ae60d2-4ab0-42a1-ad1e-f5e37c171f23 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:26:35.587932 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-81ae60d2-4ab0-42a1-ad1e-f5e37c171f23 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:26:35.602274 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ae60d2-4ab0-42a1-ad1e-f5e37c171f23 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:26:35.604761 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ae60d2-4ab0-42a1-ad1e-f5e37c171f23 service placement] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:26:35.615893 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1228/2434] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:26:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:26:39.116864 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e355b-e1b6-40c7-93d6-da4f7f79201c service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:26:39.118747 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-217e355b-e1b6-40c7-93d6-da4f7f79201c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:26:39.192457 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e355b-e1b6-40c7-93d6-da4f7f79201c service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:26:39.198585 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9807eae-1413-4266-afc4-61a57f15ccb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:26:39.200238 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9807eae-1413-4266-afc4-61a57f15ccb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:26:39.200786 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9807eae-1413-4266-afc4-61a57f15ccb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:26:39.201319 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9807eae-1413-4266-afc4-61a57f15ccb0 admin admin] PATH_INFO: `/v3/users/1e415e30c70d441991187b13c229ee18` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:26:39.203437 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9807eae-1413-4266-afc4-61a57f15ccb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e415e30c70d441991187b13c229ee18)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:26:39.205838 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9807eae-1413-4266-afc4-61a57f15ccb0 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:26:39.230374 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929f8e2-ed29-4129-865b-41f18836bc61 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:26:39.231358 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929f8e2-ed29-4129-865b-41f18836bc61 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:26:39.232086 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929f8e2-ed29-4129-865b-41f18836bc61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:26:39.251219 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-1929f8e2-ed29-4129-865b-41f18836bc61 None None] MFA Rules not processed for user `875140cf51d9410da45f8c3805b26204`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:26:39.252471 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-1929f8e2-ed29-4129-865b-41f18836bc61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:26:39.259869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1229/2435] 10.222.0.41 () {58 vars in 1009 bytes} [Fri Jun 3 16:26:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:26:39.261272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d9807eae-1413-4266-afc4-61a57f15ccb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e415e30c70d441991187b13c229ee18'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:26:39.270072 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d9807eae-1413-4266-afc4-61a57f15ccb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e415e30c70d441991187b13c229ee18'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:26:39.272703 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1929f8e2-ed29-4129-865b-41f18836bc61 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:26:39.284677 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d9807eae-1413-4266-afc4-61a57f15ccb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e415e30c70d441991187b13c229ee18'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:26:39.285213 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1929f8e2-ed29-4129-865b-41f18836bc61 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:26:39.285360 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d9807eae-1413-4266-afc4-61a57f15ccb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e415e30c70d441991187b13c229ee18'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:26:39.290826 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d45f3-9de4-450a-9a02-61136db20f96 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5f513b781a4a73b443cded13359eda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:26:39.291778 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d45f3-9de4-450a-9a02-61136db20f96 service swift] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:26:39.292068 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d45f3-9de4-450a-9a02-61136db20f96 service swift] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:26:39.292351 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d45f3-9de4-450a-9a02-61136db20f96 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:26:39.293247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d9807eae-1413-4266-afc4-61a57f15ccb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e415e30c70d441991187b13c229ee18'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:26:39.294163 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-e93d45f3-9de4-450a-9a02-61136db20f96 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:26:39.299876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1207/2436] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:26:39 2022] DELETE /identity/v3/users/1e415e30c70d441991187b13c229ee18 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:26:39.308756 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93d45f3-9de4-450a-9a02-61136db20f96 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:26:39.310371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93d45f3-9de4-450a-9a02-61136db20f96 service swift] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:26:39.318955 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1230/2437] 10.222.0.41 () {60 vars in 1384 bytes} [Fri Jun 3 16:26:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 16:26:39.538302 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9807eae-1413-4266-afc4-61a57f15ccb0 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:26:39.539997 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d9807eae-1413-4266-afc4-61a57f15ccb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:26:39.555629 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:26:39.683727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9807eae-1413-4266-afc4-61a57f15ccb0 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:26:39.688675 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69bb967-ebe3-459d-95bb-890fe544062c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:26:39.689531 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69bb967-ebe3-459d-95bb-890fe544062c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:26:39.689818 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69bb967-ebe3-459d-95bb-890fe544062c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:26:39.690100 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69bb967-ebe3-459d-95bb-890fe544062c admin admin] PATH_INFO: `/v3/projects/566b367dc7504d3091558cfd6f35bfa9` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:26:39.704023 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69bb967-ebe3-459d-95bb-890fe544062c admin admin] RBAC: Authorizing `identity:delete_project(project_id=566b367dc7504d3091558cfd6f35bfa9)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:26:39.706014 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69bb967-ebe3-459d-95bb-890fe544062c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:26:39.788089 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b69bb967-ebe3-459d-95bb-890fe544062c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '566b367dc7504d3091558cfd6f35bfa9', 'request_id': 'req-b69bb967-ebe3-459d-95bb-890fe544062c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:26:39.801546 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b69bb967-ebe3-459d-95bb-890fe544062c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '566b367dc7504d3091558cfd6f35bfa9', 'request_id': 'req-b69bb967-ebe3-459d-95bb-890fe544062c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:26:39.810486 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b69bb967-ebe3-459d-95bb-890fe544062c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '566b367dc7504d3091558cfd6f35bfa9', 'request_id': 'req-b69bb967-ebe3-459d-95bb-890fe544062c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:26:39.824385 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b69bb967-ebe3-459d-95bb-890fe544062c admin admin] The token cache is being invalidate because project 566b367dc7504d3091558cfd6f35bfa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:26:39.824834 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b69bb967-ebe3-459d-95bb-890fe544062c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:26:39.825672 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b69bb967-ebe3-459d-95bb-890fe544062c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:26:39.827568 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1208/2438] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:26:39 2022] DELETE /identity/v3/projects/566b367dc7504d3091558cfd6f35bfa9 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:26:39.838813 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d45f3-9de4-450a-9a02-61136db20f96 service swift] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:26:39.850233 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d45f3-9de4-450a-9a02-61136db20f96 service swift] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:26:39.892572 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:26:39.913742 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696991d1-ecec-46be-bb6d-a2034f5f4c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:26:39.914627 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696991d1-ecec-46be-bb6d-a2034f5f4c8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:26:39.914918 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696991d1-ecec-46be-bb6d-a2034f5f4c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:26:39.915280 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696991d1-ecec-46be-bb6d-a2034f5f4c8e admin admin] PATH_INFO: `/v3/users/0d93485016824928ab7d2550b290c0c7` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:26:39.916505 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696991d1-ecec-46be-bb6d-a2034f5f4c8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d93485016824928ab7d2550b290c0c7)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:26:39.918441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696991d1-ecec-46be-bb6d-a2034f5f4c8e admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:26:39.968942 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-696991d1-ecec-46be-bb6d-a2034f5f4c8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d93485016824928ab7d2550b290c0c7'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:26:39.976354 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-696991d1-ecec-46be-bb6d-a2034f5f4c8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d93485016824928ab7d2550b290c0c7'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:26:39.988491 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-696991d1-ecec-46be-bb6d-a2034f5f4c8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d93485016824928ab7d2550b290c0c7'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:26:39.995961 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-696991d1-ecec-46be-bb6d-a2034f5f4c8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d93485016824928ab7d2550b290c0c7'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:26:39.996813 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-696991d1-ecec-46be-bb6d-a2034f5f4c8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d93485016824928ab7d2550b290c0c7'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:26:40.002631 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1231/2439] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:26:39 2022] DELETE /identity/v3/users/0d93485016824928ab7d2550b290c0c7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:26:40.185175 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69bb967-ebe3-459d-95bb-890fe544062c admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:26:40.186724 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b69bb967-ebe3-459d-95bb-890fe544062c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:26:40.204399 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69bb967-ebe3-459d-95bb-890fe544062c admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:26:40.272154 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f9c97-0167-4d24-baaf-99f707459652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:26:40.273050 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f9c97-0167-4d24-baaf-99f707459652 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:26:40.273337 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f9c97-0167-4d24-baaf-99f707459652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:26:40.273612 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f9c97-0167-4d24-baaf-99f707459652 admin admin] PATH_INFO: `/v3/projects/ecea87a215f24b0abcf9d18797a25521` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:26:40.274953 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f9c97-0167-4d24-baaf-99f707459652 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecea87a215f24b0abcf9d18797a25521)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:26:40.276903 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f9c97-0167-4d24-baaf-99f707459652 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:26:40.320037 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:26:40.328771 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-8f5f9c97-0167-4d24-baaf-99f707459652 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecea87a215f24b0abcf9d18797a25521', 'request_id': 'req-8f5f9c97-0167-4d24-baaf-99f707459652'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:26:40.335101 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-8f5f9c97-0167-4d24-baaf-99f707459652 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecea87a215f24b0abcf9d18797a25521', 'request_id': 'req-8f5f9c97-0167-4d24-baaf-99f707459652'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:26:40.342906 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-8f5f9c97-0167-4d24-baaf-99f707459652 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecea87a215f24b0abcf9d18797a25521', 'request_id': 'req-8f5f9c97-0167-4d24-baaf-99f707459652'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:26:40.353847 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-8f5f9c97-0167-4d24-baaf-99f707459652 admin admin] The token cache is being invalidate because project ecea87a215f24b0abcf9d18797a25521 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:26:40.354139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-8f5f9c97-0167-4d24-baaf-99f707459652 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:26:40.354847 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-8f5f9c97-0167-4d24-baaf-99f707459652 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:26:40.356733 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1209/2440] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:26:40 2022] DELETE /identity/v3/projects/ecea87a215f24b0abcf9d18797a25521 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:26:49.999148 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=34,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:26:50.350185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=10,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:27:03.407028 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696991d1-ecec-46be-bb6d-a2034f5f4c8e admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:03.409075 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-696991d1-ecec-46be-bb6d-a2034f5f4c8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:27:03.505349 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696991d1-ecec-46be-bb6d-a2034f5f4c8e admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:03.569658 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530954c1-ee1c-4479-bf7f-1ac594a4c3d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:03.570309 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530954c1-ee1c-4479-bf7f-1ac594a4c3d3 service nova] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:03.570601 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530954c1-ee1c-4479-bf7f-1ac594a4c3d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:03.570877 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530954c1-ee1c-4479-bf7f-1ac594a4c3d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:03.572695 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-530954c1-ee1c-4479-bf7f-1ac594a4c3d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:27:03.584147 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530954c1-ee1c-4479-bf7f-1ac594a4c3d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:03.585711 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530954c1-ee1c-4479-bf7f-1ac594a4c3d3 service nova] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:03.596751 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1232/2441] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:27:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:27:04.686347 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f9c97-0167-4d24-baaf-99f707459652 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:04.687893 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8f5f9c97-0167-4d24-baaf-99f707459652 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:27:04.703982 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f9c97-0167-4d24-baaf-99f707459652 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:04.710061 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681991db-36d8-4edf-9e97-9271f420675f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:04.710708 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681991db-36d8-4edf-9e97-9271f420675f service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:04.710999 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681991db-36d8-4edf-9e97-9271f420675f service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:04.711275 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681991db-36d8-4edf-9e97-9271f420675f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:04.713101 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-681991db-36d8-4edf-9e97-9271f420675f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:27:04.790175 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681991db-36d8-4edf-9e97-9271f420675f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:04.791757 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681991db-36d8-4edf-9e97-9271f420675f service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:04.809264 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1210/2442] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:27:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:27:05.637486 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93865b6-777d-43fe-9863-a306e7d3f461 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:05.637953 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93865b6-777d-43fe-9863-a306e7d3f461 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:05.638067 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93865b6-777d-43fe-9863-a306e7d3f461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:05.655544 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-e93865b6-777d-43fe-9863-a306e7d3f461 None None] MFA Rules not processed for user `875140cf51d9410da45f8c3805b26204`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:27:05.656724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-e93865b6-777d-43fe-9863-a306e7d3f461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:27:05.709623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1233/2443] 10.222.0.41 () {58 vars in 967 bytes} [Fri Jun 3 16:27:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:27:05.721974 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681991db-36d8-4edf-9e97-9271f420675f service nova] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:05.723428 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-681991db-36d8-4edf-9e97-9271f420675f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:27:05.793222 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681991db-36d8-4edf-9e97-9271f420675f service nova] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:05.798471 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ff393-e434-4d42-b661-b39cd0196493 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5f513b781a4a73b443cded13359eda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:05.799200 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ff393-e434-4d42-b661-b39cd0196493 service swift] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:05.799508 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ff393-e434-4d42-b661-b39cd0196493 service swift] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:05.799817 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ff393-e434-4d42-b661-b39cd0196493 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:05.808303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ff393-e434-4d42-b661-b39cd0196493 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:05.809933 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ff393-e434-4d42-b661-b39cd0196493 service swift] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:05.818261 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1211/2444] 10.222.0.41 () {60 vars in 1384 bytes} [Fri Jun 3 16:27:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 16:27:15.703145 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=87 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:27:15.815275 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=73 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:27:37.089372 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93865b6-777d-43fe-9863-a306e7d3f461 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:37.090785 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-e93865b6-777d-43fe-9863-a306e7d3f461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:27:37.147033 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93865b6-777d-43fe-9863-a306e7d3f461 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:37.150843 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a35c6-d3d3-47fc-a460-9afeb2f5976e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:37.151524 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a35c6-d3d3-47fc-a460-9afeb2f5976e service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:37.151810 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a35c6-d3d3-47fc-a460-9afeb2f5976e service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:37.152088 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a35c6-d3d3-47fc-a460-9afeb2f5976e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:37.153861 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-522a35c6-d3d3-47fc-a460-9afeb2f5976e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:27:37.213942 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522a35c6-d3d3-47fc-a460-9afeb2f5976e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:37.215388 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522a35c6-d3d3-47fc-a460-9afeb2f5976e service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:37.224895 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1234/2445] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:27:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:27:40.261152 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ff393-e434-4d42-b661-b39cd0196493 service swift] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:40.262768 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6f7ff393-e434-4d42-b661-b39cd0196493 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:27:40.280912 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ff393-e434-4d42-b661-b39cd0196493 service swift] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:40.286340 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b716988b-3065-41fe-ab3c-37435f0951c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:40.287560 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716988b-3065-41fe-ab3c-37435f0951c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:40.287860 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716988b-3065-41fe-ab3c-37435f0951c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:40.288145 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716988b-3065-41fe-ab3c-37435f0951c6 admin admin] PATH_INFO: `/v3/users/c68189df1f9b4f9aacfa319381849697` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:40.289378 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b716988b-3065-41fe-ab3c-37435f0951c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c68189df1f9b4f9aacfa319381849697)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:40.291766 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b716988b-3065-41fe-ab3c-37435f0951c6 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:40.344098 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b716988b-3065-41fe-ab3c-37435f0951c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c68189df1f9b4f9aacfa319381849697'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:27:40.356654 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b716988b-3065-41fe-ab3c-37435f0951c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c68189df1f9b4f9aacfa319381849697'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:27:40.368766 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b716988b-3065-41fe-ab3c-37435f0951c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c68189df1f9b4f9aacfa319381849697'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:27:40.369402 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b716988b-3065-41fe-ab3c-37435f0951c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c68189df1f9b4f9aacfa319381849697'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:27:40.375968 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b716988b-3065-41fe-ab3c-37435f0951c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c68189df1f9b4f9aacfa319381849697'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:27:40.381747 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1212/2446] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:27:40 2022] DELETE /identity/v3/users/c68189df1f9b4f9aacfa319381849697 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:27:40.557967 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a35c6-d3d3-47fc-a460-9afeb2f5976e service neutron] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:40.568493 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a35c6-d3d3-47fc-a460-9afeb2f5976e service neutron] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:40.624942 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff1630-1f96-4953-bf77-5f8215cdd006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:40.625733 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff1630-1f96-4953-bf77-5f8215cdd006 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:40.626014 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff1630-1f96-4953-bf77-5f8215cdd006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:40.626289 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff1630-1f96-4953-bf77-5f8215cdd006 admin admin] PATH_INFO: `/v3/projects/d0f59694966848829048aee648dddccc` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:40.627483 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ff1630-1f96-4953-bf77-5f8215cdd006 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0f59694966848829048aee648dddccc)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:40.629384 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ff1630-1f96-4953-bf77-5f8215cdd006 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:40.674720 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-e9ff1630-1f96-4953-bf77-5f8215cdd006 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0f59694966848829048aee648dddccc', 'request_id': 'req-e9ff1630-1f96-4953-bf77-5f8215cdd006'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:27:40.676538 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:27:40.680943 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-e9ff1630-1f96-4953-bf77-5f8215cdd006 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0f59694966848829048aee648dddccc', 'request_id': 'req-e9ff1630-1f96-4953-bf77-5f8215cdd006'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:27:40.688531 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-e9ff1630-1f96-4953-bf77-5f8215cdd006 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0f59694966848829048aee648dddccc', 'request_id': 'req-e9ff1630-1f96-4953-bf77-5f8215cdd006'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:27:40.698775 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-e9ff1630-1f96-4953-bf77-5f8215cdd006 admin admin] The token cache is being invalidate because project d0f59694966848829048aee648dddccc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:27:40.699063 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-e9ff1630-1f96-4953-bf77-5f8215cdd006 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:27:40.699749 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-e9ff1630-1f96-4953-bf77-5f8215cdd006 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:27:40.701286 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1235/2447] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:27:40 2022] DELETE /identity/v3/projects/d0f59694966848829048aee648dddccc => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:27:40.706752 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502847c-e975-44a1-9da6-fb5486da9586 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:40.707045 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502847c-e975-44a1-9da6-fb5486da9586 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:40.707356 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502847c-e975-44a1-9da6-fb5486da9586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:40.724881 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-2502847c-e975-44a1-9da6-fb5486da9586 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:27:40.726120 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2502847c-e975-44a1-9da6-fb5486da9586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:27:40.785056 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1213/2448] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:27:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:27:40.792938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336e17f-6623-4f23-b0c2-29283b5e5553 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:40.793227 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336e17f-6623-4f23-b0c2-29283b5e5553 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:40.793629 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336e17f-6623-4f23-b0c2-29283b5e5553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:40.811651 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-d336e17f-6623-4f23-b0c2-29283b5e5553 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:27:40.812876 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-d336e17f-6623-4f23-b0c2-29283b5e5553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:27:40.820542 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1236/2449] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:27:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:27:40.824473 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2502847c-e975-44a1-9da6-fb5486da9586 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:40.834576 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2502847c-e975-44a1-9da6-fb5486da9586 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:40.839128 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b3af13-0bb5-4db3-a28b-33fbf6f74321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:40.839804 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3af13-0bb5-4db3-a28b-33fbf6f74321 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:40.840093 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3af13-0bb5-4db3-a28b-33fbf6f74321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:40.840376 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3af13-0bb5-4db3-a28b-33fbf6f74321 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:40.841131 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b3af13-0bb5-4db3-a28b-33fbf6f74321 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:27:40.841442 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b3af13-0bb5-4db3-a28b-33fbf6f74321 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:40.843100 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b3af13-0bb5-4db3-a28b-33fbf6f74321 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:40.854343 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1214/2450] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:27:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:27:40.858353 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336e17f-6623-4f23-b0c2-29283b5e5553 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:40.867419 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336e17f-6623-4f23-b0c2-29283b5e5553 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:40.871987 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb12cc6-ab8e-44f5-a71f-b317f56b5d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:40.872624 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb12cc6-ab8e-44f5-a71f-b317f56b5d46 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:40.872906 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb12cc6-ab8e-44f5-a71f-b317f56b5d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:40.873180 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb12cc6-ab8e-44f5-a71f-b317f56b5d46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:40.874368 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb12cc6-ab8e-44f5-a71f-b317f56b5d46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:40.876299 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb12cc6-ab8e-44f5-a71f-b317f56b5d46 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:40.901170 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1237/2451] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:27:40 2022] POST /identity/v3/projects => generated 405 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:27:40.905141 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b3af13-0bb5-4db3-a28b-33fbf6f74321 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:40.914872 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b3af13-0bb5-4db3-a28b-33fbf6f74321 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:40.969400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea474c-3b06-414b-b959-ccb32f3185d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:40.970058 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea474c-3b06-414b-b959-ccb32f3185d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:40.970344 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea474c-3b06-414b-b959-ccb32f3185d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:40.970622 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea474c-3b06-414b-b959-ccb32f3185d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:40.971345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea474c-3b06-414b-b959-ccb32f3185d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:40.973265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea474c-3b06-414b-b959-ccb32f3185d3 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:40.979830 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:27:40.995119 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1215/2452] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:27:40 2022] POST /identity/v3/users => generated 303 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:27:40.999800 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb12cc6-ab8e-44f5-a71f-b317f56b5d46 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:41.009742 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb12cc6-ab8e-44f5-a71f-b317f56b5d46 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:41.014234 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b94970b-c07f-4269-ad4a-5ecc3b1974b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:41.014879 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94970b-c07f-4269-ad4a-5ecc3b1974b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:41.015161 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94970b-c07f-4269-ad4a-5ecc3b1974b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:41.015474 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94970b-c07f-4269-ad4a-5ecc3b1974b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:41.016188 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b94970b-c07f-4269-ad4a-5ecc3b1974b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:41.017762 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b94970b-c07f-4269-ad4a-5ecc3b1974b4 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:41.027995 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1238/2453] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:27:41.031593 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea474c-3b06-414b-b959-ccb32f3185d3 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:41.041300 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea474c-3b06-414b-b959-ccb32f3185d3 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:41.046452 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab75f23-b61d-4b74-955c-cd8bd152eff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:41.047129 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab75f23-b61d-4b74-955c-cd8bd152eff1 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:41.047443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab75f23-b61d-4b74-955c-cd8bd152eff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:41.047717 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab75f23-b61d-4b74-955c-cd8bd152eff1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:41.048434 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab75f23-b61d-4b74-955c-cd8bd152eff1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:41.050032 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab75f23-b61d-4b74-955c-cd8bd152eff1 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:41.059816 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1216/2454] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:27:41.064103 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b94970b-c07f-4269-ad4a-5ecc3b1974b4 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:41.074154 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b94970b-c07f-4269-ad4a-5ecc3b1974b4 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:41.079361 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb304d4b-463f-44f2-a4d1-ace35edd9fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:41.080235 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb304d4b-463f-44f2-a4d1-ace35edd9fe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:41.080524 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb304d4b-463f-44f2-a4d1-ace35edd9fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:41.080807 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb304d4b-463f-44f2-a4d1-ace35edd9fe5 admin admin] PATH_INFO: `/v3/projects/8b4de93af39b4952bfaa5577b473eff4/users/7e77d233ef724103b90d2c4545b3dad7/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:41.095818 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb304d4b-463f-44f2-a4d1-ace35edd9fe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b4de93af39b4952bfaa5577b473eff4, user_id=7e77d233ef724103b90d2c4545b3dad7, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:41.098456 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb304d4b-463f-44f2-a4d1-ace35edd9fe5 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:41.110874 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1239/2455] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:27:41 2022] PUT /identity/v3/projects/8b4de93af39b4952bfaa5577b473eff4/users/7e77d233ef724103b90d2c4545b3dad7/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:27:41.116127 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccbe15-7d69-4a18-9d06-c6c90bca8685 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:41.116425 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccbe15-7d69-4a18-9d06-c6c90bca8685 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:41.116703 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccbe15-7d69-4a18-9d06-c6c90bca8685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:41.134416 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-0dccbe15-7d69-4a18-9d06-c6c90bca8685 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:27:41.135718 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0dccbe15-7d69-4a18-9d06-c6c90bca8685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:27:41.193593 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1217/2456] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:27:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:27:41.203611 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb304d4b-463f-44f2-a4d1-ace35edd9fe5 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:41.205291 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-eb304d4b-463f-44f2-a4d1-ace35edd9fe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:27:41.260838 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb304d4b-463f-44f2-a4d1-ace35edd9fe5 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:41.265060 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a637166-5cfe-4f6d-a0af-7d5eee5961ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:41.265931 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a637166-5cfe-4f6d-a0af-7d5eee5961ee service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:41.266213 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a637166-5cfe-4f6d-a0af-7d5eee5961ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:41.266488 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a637166-5cfe-4f6d-a0af-7d5eee5961ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:41.275802 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a637166-5cfe-4f6d-a0af-7d5eee5961ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:41.277208 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a637166-5cfe-4f6d-a0af-7d5eee5961ee service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:41.286596 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1240/2457] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:27:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:27:41.882326 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b4ef9-95d2-40ba-a61e-f55f912d36cc None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:41.882818 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b4ef9-95d2-40ba-a61e-f55f912d36cc None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:41.883072 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b4ef9-95d2-40ba-a61e-f55f912d36cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:41.911501 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-f56b4ef9-95d2-40ba-a61e-f55f912d36cc None None] MFA Rules not processed for user `7e77d233ef724103b90d2c4545b3dad7`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:27:41.912715 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f56b4ef9-95d2-40ba-a61e-f55f912d36cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:27:41.980052 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1218/2458] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:27:41 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:27:41.984474 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a637166-5cfe-4f6d-a0af-7d5eee5961ee service neutron] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:41.994162 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a637166-5cfe-4f6d-a0af-7d5eee5961ee service neutron] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:42.000100 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2036430a-8db2-4d73-a37a-21c95e2a0d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:42.000748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036430a-8db2-4d73-a37a-21c95e2a0d9d admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:42.001030 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036430a-8db2-4d73-a37a-21c95e2a0d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:42.001303 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036430a-8db2-4d73-a37a-21c95e2a0d9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:42.001995 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2036430a-8db2-4d73-a37a-21c95e2a0d9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:42.003906 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2036430a-8db2-4d73-a37a-21c95e2a0d9d admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:42.033653 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1241/2459] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:27:41 2022] POST /identity/v3/projects => generated 403 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:27:42.035060 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b4ef9-95d2-40ba-a61e-f55f912d36cc None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:42.045503 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b4ef9-95d2-40ba-a61e-f55f912d36cc None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:42.101777 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9279b769-0221-47a7-90aa-dd2fef0322de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:42.102365 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279b769-0221-47a7-90aa-dd2fef0322de admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:42.102650 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279b769-0221-47a7-90aa-dd2fef0322de admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:42.102929 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279b769-0221-47a7-90aa-dd2fef0322de admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:42.103664 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9279b769-0221-47a7-90aa-dd2fef0322de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:42.105608 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9279b769-0221-47a7-90aa-dd2fef0322de admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:42.119727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:27:42.129826 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1219/2460] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:27:42 2022] POST /identity/v3/users => generated 308 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:27:42.130899 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2036430a-8db2-4d73-a37a-21c95e2a0d9d admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:42.141711 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2036430a-8db2-4d73-a37a-21c95e2a0d9d admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:42.146269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5b96c-11b2-4389-80db-4a0421e488c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:42.146923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5b96c-11b2-4389-80db-4a0421e488c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:42.147219 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5b96c-11b2-4389-80db-4a0421e488c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:42.147516 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5b96c-11b2-4389-80db-4a0421e488c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:42.148701 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c5b96c-11b2-4389-80db-4a0421e488c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:42.150108 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c5b96c-11b2-4389-80db-4a0421e488c8 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:42.161075 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1242/2461] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:27:42.165150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9279b769-0221-47a7-90aa-dd2fef0322de admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:42.175168 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9279b769-0221-47a7-90aa-dd2fef0322de admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:42.179833 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7221e-8ce6-46cd-9ad3-640167db3be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:42.180714 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7221e-8ce6-46cd-9ad3-640167db3be5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:42.181007 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7221e-8ce6-46cd-9ad3-640167db3be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:42.181294 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7221e-8ce6-46cd-9ad3-640167db3be5 admin admin] PATH_INFO: `/v3/projects/ce18c25adf1e4bf9b8da6f03486d5f81/users/1ce9e3e6625947e19ec123b47136e7de/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:42.196672 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed7221e-8ce6-46cd-9ad3-640167db3be5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce18c25adf1e4bf9b8da6f03486d5f81, user_id=1ce9e3e6625947e19ec123b47136e7de, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:42.199358 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed7221e-8ce6-46cd-9ad3-640167db3be5 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:42.212019 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1220/2462] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:27:42 2022] PUT /identity/v3/projects/ce18c25adf1e4bf9b8da6f03486d5f81/users/1ce9e3e6625947e19ec123b47136e7de/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:27:42.215838 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5b96c-11b2-4389-80db-4a0421e488c8 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:42.226780 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5b96c-11b2-4389-80db-4a0421e488c8 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:42.269126 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:27:42.283942 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbed6c2-491f-478b-8323-83a76fd21f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:42.284622 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbed6c2-491f-478b-8323-83a76fd21f09 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:42.284924 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbed6c2-491f-478b-8323-83a76fd21f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:42.285197 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbed6c2-491f-478b-8323-83a76fd21f09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:42.285911 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbed6c2-491f-478b-8323-83a76fd21f09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:42.287541 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbed6c2-491f-478b-8323-83a76fd21f09 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:42.297577 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1243/2463] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:27:42.301765 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7221e-8ce6-46cd-9ad3-640167db3be5 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:42.315054 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7221e-8ce6-46cd-9ad3-640167db3be5 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:42.319817 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e214380-6496-4b41-9a43-789554c2f999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:42.320736 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e214380-6496-4b41-9a43-789554c2f999 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:42.321024 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e214380-6496-4b41-9a43-789554c2f999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:42.321305 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e214380-6496-4b41-9a43-789554c2f999 admin admin] PATH_INFO: `/v3/projects/ce18c25adf1e4bf9b8da6f03486d5f81/users/1ce9e3e6625947e19ec123b47136e7de/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:42.323277 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e214380-6496-4b41-9a43-789554c2f999 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce18c25adf1e4bf9b8da6f03486d5f81, user_id=1ce9e3e6625947e19ec123b47136e7de, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:42.325983 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e214380-6496-4b41-9a43-789554c2f999 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:42.336343 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1221/2464] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:27:42 2022] PUT /identity/v3/projects/ce18c25adf1e4bf9b8da6f03486d5f81/users/1ce9e3e6625947e19ec123b47136e7de/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:27:42.955348 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77afc61-1b39-4080-a681-ab823409804e None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:42.955655 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77afc61-1b39-4080-a681-ab823409804e None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:42.955942 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77afc61-1b39-4080-a681-ab823409804e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:42.982958 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-d77afc61-1b39-4080-a681-ab823409804e None None] MFA Rules not processed for user `1ce9e3e6625947e19ec123b47136e7de`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:27:42.984892 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-d77afc61-1b39-4080-a681-ab823409804e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:27:43.054262 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1244/2465] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:27:42 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:27:44.035940 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e214380-6496-4b41-9a43-789554c2f999 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:44.097164 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e214380-6496-4b41-9a43-789554c2f999 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:44.101828 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec01c5-288a-461a-bff2-1363f02882ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:44.102691 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec01c5-288a-461a-bff2-1363f02882ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:44.102974 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec01c5-288a-461a-bff2-1363f02882ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:44.103250 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec01c5-288a-461a-bff2-1363f02882ae admin admin] PATH_INFO: `/v3/users/7e77d233ef724103b90d2c4545b3dad7` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:44.104455 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ec01c5-288a-461a-bff2-1363f02882ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e77d233ef724103b90d2c4545b3dad7)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:44.106370 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ec01c5-288a-461a-bff2-1363f02882ae admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:44.152394 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e9ec01c5-288a-461a-bff2-1363f02882ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e77d233ef724103b90d2c4545b3dad7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:27:44.159806 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e9ec01c5-288a-461a-bff2-1363f02882ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e77d233ef724103b90d2c4545b3dad7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:27:44.172499 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e9ec01c5-288a-461a-bff2-1363f02882ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e77d233ef724103b90d2c4545b3dad7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:27:44.173131 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e9ec01c5-288a-461a-bff2-1363f02882ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e77d233ef724103b90d2c4545b3dad7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:27:44.179876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e9ec01c5-288a-461a-bff2-1363f02882ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e77d233ef724103b90d2c4545b3dad7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:27:44.185805 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1222/2466] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:27:44 2022] DELETE /identity/v3/users/7e77d233ef724103b90d2c4545b3dad7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:27:44.353771 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77afc61-1b39-4080-a681-ab823409804e None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:44.370312 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77afc61-1b39-4080-a681-ab823409804e None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:44.438945 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cbfa13-baaa-4e45-804d-e3e233a5a2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:44.439765 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbfa13-baaa-4e45-804d-e3e233a5a2b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:44.440048 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbfa13-baaa-4e45-804d-e3e233a5a2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:44.440352 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbfa13-baaa-4e45-804d-e3e233a5a2b1 admin admin] PATH_INFO: `/v3/projects/8b4de93af39b4952bfaa5577b473eff4` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:44.441513 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cbfa13-baaa-4e45-804d-e3e233a5a2b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b4de93af39b4952bfaa5577b473eff4)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:44.443443 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cbfa13-baaa-4e45-804d-e3e233a5a2b1 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:44.487650 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-b6cbfa13-baaa-4e45-804d-e3e233a5a2b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b4de93af39b4952bfaa5577b473eff4', 'request_id': 'req-b6cbfa13-baaa-4e45-804d-e3e233a5a2b1'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:27:44.493495 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-b6cbfa13-baaa-4e45-804d-e3e233a5a2b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b4de93af39b4952bfaa5577b473eff4', 'request_id': 'req-b6cbfa13-baaa-4e45-804d-e3e233a5a2b1'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:27:44.500614 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-b6cbfa13-baaa-4e45-804d-e3e233a5a2b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b4de93af39b4952bfaa5577b473eff4', 'request_id': 'req-b6cbfa13-baaa-4e45-804d-e3e233a5a2b1'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:27:44.506958 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:27:44.512246 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-b6cbfa13-baaa-4e45-804d-e3e233a5a2b1 admin admin] The token cache is being invalidate because project 8b4de93af39b4952bfaa5577b473eff4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:27:44.512573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-b6cbfa13-baaa-4e45-804d-e3e233a5a2b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:27:44.513423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-b6cbfa13-baaa-4e45-804d-e3e233a5a2b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:27:44.515554 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1245/2467] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:27:44 2022] DELETE /identity/v3/projects/8b4de93af39b4952bfaa5577b473eff4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:27:44.519446 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec01c5-288a-461a-bff2-1363f02882ae admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:44.520846 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e9ec01c5-288a-461a-bff2-1363f02882ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:27:44.534096 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec01c5-288a-461a-bff2-1363f02882ae admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:44.571553 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:27:44.590781 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b3b70e-68aa-4386-869e-639855662f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:44.591782 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b3b70e-68aa-4386-869e-639855662f12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:44.592068 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b3b70e-68aa-4386-869e-639855662f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:44.592346 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b3b70e-68aa-4386-869e-639855662f12 admin admin] PATH_INFO: `/v3/users/1ce9e3e6625947e19ec123b47136e7de` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:44.594026 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b3b70e-68aa-4386-869e-639855662f12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ce9e3e6625947e19ec123b47136e7de)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:44.595980 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b3b70e-68aa-4386-869e-639855662f12 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:44.640418 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d7b3b70e-68aa-4386-869e-639855662f12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ce9e3e6625947e19ec123b47136e7de'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:27:44.649950 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d7b3b70e-68aa-4386-869e-639855662f12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ce9e3e6625947e19ec123b47136e7de'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:27:44.662681 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d7b3b70e-68aa-4386-869e-639855662f12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ce9e3e6625947e19ec123b47136e7de'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:27:44.663319 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d7b3b70e-68aa-4386-869e-639855662f12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ce9e3e6625947e19ec123b47136e7de'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:27:44.670626 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d7b3b70e-68aa-4386-869e-639855662f12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ce9e3e6625947e19ec123b47136e7de'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:27:44.681724 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1223/2468] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:27:44 2022] DELETE /identity/v3/users/1ce9e3e6625947e19ec123b47136e7de => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:27:44.860000 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cbfa13-baaa-4e45-804d-e3e233a5a2b1 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:44.861782 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-b6cbfa13-baaa-4e45-804d-e3e233a5a2b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:27:44.876711 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cbfa13-baaa-4e45-804d-e3e233a5a2b1 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:44.932526 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e6d7c-826a-4f7d-a8da-51ea9950f814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:44.933383 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e6d7c-826a-4f7d-a8da-51ea9950f814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:44.933670 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e6d7c-826a-4f7d-a8da-51ea9950f814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:44.933952 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e6d7c-826a-4f7d-a8da-51ea9950f814 admin admin] PATH_INFO: `/v3/projects/ce18c25adf1e4bf9b8da6f03486d5f81` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:44.935124 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542e6d7c-826a-4f7d-a8da-51ea9950f814 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce18c25adf1e4bf9b8da6f03486d5f81)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:44.937096 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542e6d7c-826a-4f7d-a8da-51ea9950f814 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:44.983489 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-542e6d7c-826a-4f7d-a8da-51ea9950f814 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce18c25adf1e4bf9b8da6f03486d5f81', 'request_id': 'req-542e6d7c-826a-4f7d-a8da-51ea9950f814'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:27:44.989413 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-542e6d7c-826a-4f7d-a8da-51ea9950f814 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce18c25adf1e4bf9b8da6f03486d5f81', 'request_id': 'req-542e6d7c-826a-4f7d-a8da-51ea9950f814'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:27:44.996603 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-542e6d7c-826a-4f7d-a8da-51ea9950f814 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce18c25adf1e4bf9b8da6f03486d5f81', 'request_id': 'req-542e6d7c-826a-4f7d-a8da-51ea9950f814'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:27:45.016301 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-542e6d7c-826a-4f7d-a8da-51ea9950f814 admin admin] The token cache is being invalidate because project ce18c25adf1e4bf9b8da6f03486d5f81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:27:45.016616 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-542e6d7c-826a-4f7d-a8da-51ea9950f814 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:27:45.017345 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-542e6d7c-826a-4f7d-a8da-51ea9950f814 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:27:45.018950 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1246/2469] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:27:44 2022] DELETE /identity/v3/projects/ce18c25adf1e4bf9b8da6f03486d5f81 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:27:45.023868 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41ff93-8c01-424f-92e8-1482e9782354 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:45.024247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41ff93-8c01-424f-92e8-1482e9782354 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:45.024842 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41ff93-8c01-424f-92e8-1482e9782354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:45.044264 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-9d41ff93-8c01-424f-92e8-1482e9782354 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:27:45.045743 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9d41ff93-8c01-424f-92e8-1482e9782354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:27:45.105807 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1224/2470] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:27:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:27:45.113445 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a2fb1a-cfd6-4db2-8865-c1e50c5a043e None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:45.113736 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a2fb1a-cfd6-4db2-8865-c1e50c5a043e None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:45.114014 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a2fb1a-cfd6-4db2-8865-c1e50c5a043e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:45.133079 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-41a2fb1a-cfd6-4db2-8865-c1e50c5a043e None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:27:45.134289 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-41a2fb1a-cfd6-4db2-8865-c1e50c5a043e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:27:45.142837 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1247/2471] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:27:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:27:45.146910 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41ff93-8c01-424f-92e8-1482e9782354 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:45.156699 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41ff93-8c01-424f-92e8-1482e9782354 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:45.161266 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00504777-caf5-4931-91cd-c2286a6e7870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:45.161904 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00504777-caf5-4931-91cd-c2286a6e7870 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:45.162188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00504777-caf5-4931-91cd-c2286a6e7870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:45.162460 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00504777-caf5-4931-91cd-c2286a6e7870 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:45.163180 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00504777-caf5-4931-91cd-c2286a6e7870 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:27:45.163518 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00504777-caf5-4931-91cd-c2286a6e7870 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:45.165118 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00504777-caf5-4931-91cd-c2286a6e7870 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:45.176873 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1225/2472] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:27:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:27:45.181083 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a2fb1a-cfd6-4db2-8865-c1e50c5a043e None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:45.190603 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a2fb1a-cfd6-4db2-8865-c1e50c5a043e None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:45.195214 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60393155-b606-4d60-8d8e-a6d9befb6d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:45.195904 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60393155-b606-4d60-8d8e-a6d9befb6d4e admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:45.196189 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60393155-b606-4d60-8d8e-a6d9befb6d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:45.196464 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60393155-b606-4d60-8d8e-a6d9befb6d4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:45.197145 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60393155-b606-4d60-8d8e-a6d9befb6d4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:45.199020 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60393155-b606-4d60-8d8e-a6d9befb6d4e admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:45.226495 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1248/2473] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:27:45 2022] POST /identity/v3/projects => generated 357 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:27:45.227973 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00504777-caf5-4931-91cd-c2286a6e7870 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:45.238150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00504777-caf5-4931-91cd-c2286a6e7870 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:45.282211 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:27:45.295901 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca6794-8e57-4d49-8b26-3f4d69f0558a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:45.296572 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca6794-8e57-4d49-8b26-3f4d69f0558a admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:45.296858 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca6794-8e57-4d49-8b26-3f4d69f0558a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:45.297135 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca6794-8e57-4d49-8b26-3f4d69f0558a admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:45.297828 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ca6794-8e57-4d49-8b26-3f4d69f0558a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:45.299768 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ca6794-8e57-4d49-8b26-3f4d69f0558a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:45.319199 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1226/2474] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:27:45 2022] POST /identity/v3/users => generated 279 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:27:45.323246 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60393155-b606-4d60-8d8e-a6d9befb6d4e admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:45.333802 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60393155-b606-4d60-8d8e-a6d9befb6d4e admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:45.338746 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83819e3-5d0a-4db7-9f15-c68cee9e506e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:45.339825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83819e3-5d0a-4db7-9f15-c68cee9e506e admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:45.340286 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83819e3-5d0a-4db7-9f15-c68cee9e506e admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:45.340747 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83819e3-5d0a-4db7-9f15-c68cee9e506e admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:45.341976 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83819e3-5d0a-4db7-9f15-c68cee9e506e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:45.345076 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83819e3-5d0a-4db7-9f15-c68cee9e506e admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:45.361602 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1249/2475] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:27:45.365339 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca6794-8e57-4d49-8b26-3f4d69f0558a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:45.375260 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca6794-8e57-4d49-8b26-3f4d69f0558a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:45.380192 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f91ca3-cff5-42a4-a03d-fc003c823b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:45.380874 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f91ca3-cff5-42a4-a03d-fc003c823b92 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:45.381143 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f91ca3-cff5-42a4-a03d-fc003c823b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:45.381424 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f91ca3-cff5-42a4-a03d-fc003c823b92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:45.382155 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f91ca3-cff5-42a4-a03d-fc003c823b92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:45.383822 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f91ca3-cff5-42a4-a03d-fc003c823b92 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:45.394368 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1227/2476] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:27:45.398404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83819e3-5d0a-4db7-9f15-c68cee9e506e admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:45.409246 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83819e3-5d0a-4db7-9f15-c68cee9e506e admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:45.413872 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e58fb-e55c-4a2e-b37d-8ed0d0ac8523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:45.414747 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e58fb-e55c-4a2e-b37d-8ed0d0ac8523 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:45.415033 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e58fb-e55c-4a2e-b37d-8ed0d0ac8523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:45.415340 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e58fb-e55c-4a2e-b37d-8ed0d0ac8523 admin admin] PATH_INFO: `/v3/projects/3a764b1e21f843c7a8680fc5eed1d6dc/users/6e1074fa7fce448bb412ed47270344b3/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:45.429623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e58fb-e55c-4a2e-b37d-8ed0d0ac8523 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a764b1e21f843c7a8680fc5eed1d6dc, user_id=6e1074fa7fce448bb412ed47270344b3, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:45.432289 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e58fb-e55c-4a2e-b37d-8ed0d0ac8523 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:45.444515 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1250/2477] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:27:45 2022] PUT /identity/v3/projects/3a764b1e21f843c7a8680fc5eed1d6dc/users/6e1074fa7fce448bb412ed47270344b3/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:27:45.455074 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec84c0-c2bc-49e2-937d-f359a7f9f57f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:45.455394 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec84c0-c2bc-49e2-937d-f359a7f9f57f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:45.455676 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec84c0-c2bc-49e2-937d-f359a7f9f57f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:45.481973 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-5aec84c0-c2bc-49e2-937d-f359a7f9f57f None None] MFA Rules not processed for user `6e1074fa7fce448bb412ed47270344b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:27:45.483192 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5aec84c0-c2bc-49e2-937d-f359a7f9f57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:27:45.544919 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1228/2478] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:27:45 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:27:45.549724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e58fb-e55c-4a2e-b37d-8ed0d0ac8523 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:45.576225 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:27:45.612672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e58fb-e55c-4a2e-b37d-8ed0d0ac8523 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:45.617311 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77632e9f-ffe8-441e-8b49-599f30e90a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:45.617968 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77632e9f-ffe8-441e-8b49-599f30e90a4b admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:45.618256 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77632e9f-ffe8-441e-8b49-599f30e90a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:45.618542 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77632e9f-ffe8-441e-8b49-599f30e90a4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:45.619239 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77632e9f-ffe8-441e-8b49-599f30e90a4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:45.621179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77632e9f-ffe8-441e-8b49-599f30e90a4b admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:45.646302 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1251/2479] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:27:45 2022] POST /identity/v3/projects => generated 355 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:27:45.650641 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec84c0-c2bc-49e2-937d-f359a7f9f57f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:45.661847 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec84c0-c2bc-49e2-937d-f359a7f9f57f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:45.718771 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b8fa4-9261-4707-86ec-19d0cf11928f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:45.719469 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b8fa4-9261-4707-86ec-19d0cf11928f admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:45.719759 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b8fa4-9261-4707-86ec-19d0cf11928f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:45.720045 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b8fa4-9261-4707-86ec-19d0cf11928f admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:45.720771 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b8fa4-9261-4707-86ec-19d0cf11928f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:45.722706 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b8fa4-9261-4707-86ec-19d0cf11928f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:45.742979 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1229/2480] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:27:45 2022] POST /identity/v3/users => generated 284 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:27:45.747156 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77632e9f-ffe8-441e-8b49-599f30e90a4b admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:45.758245 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77632e9f-ffe8-441e-8b49-599f30e90a4b admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:45.763187 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90a4bf-4531-4a0e-99e2-beeb2e544198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:45.763885 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90a4bf-4531-4a0e-99e2-beeb2e544198 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:45.764188 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90a4bf-4531-4a0e-99e2-beeb2e544198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:45.764518 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90a4bf-4531-4a0e-99e2-beeb2e544198 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:45.765245 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e90a4bf-4531-4a0e-99e2-beeb2e544198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:45.766831 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e90a4bf-4531-4a0e-99e2-beeb2e544198 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:45.777767 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1252/2481] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:27:45.782414 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b8fa4-9261-4707-86ec-19d0cf11928f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:45.792381 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b8fa4-9261-4707-86ec-19d0cf11928f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:45.797691 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c789660-4183-473b-afd1-471e33170aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:45.798508 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c789660-4183-473b-afd1-471e33170aec admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:45.798789 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c789660-4183-473b-afd1-471e33170aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:45.799063 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c789660-4183-473b-afd1-471e33170aec admin admin] PATH_INFO: `/v3/projects/b750fbe5306a4484bd3305004749fdf6/users/52218b6d68d4464f8c63f748c4c5255a/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:45.813284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c789660-4183-473b-afd1-471e33170aec admin admin] RBAC: Authorizing `identity:create_grant(project_id=b750fbe5306a4484bd3305004749fdf6, user_id=52218b6d68d4464f8c63f748c4c5255a, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:45.815953 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c789660-4183-473b-afd1-471e33170aec admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:45.828183 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1230/2482] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:27:45 2022] PUT /identity/v3/projects/b750fbe5306a4484bd3305004749fdf6/users/52218b6d68d4464f8c63f748c4c5255a/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:27:45.832115 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90a4bf-4531-4a0e-99e2-beeb2e544198 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:45.841950 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90a4bf-4531-4a0e-99e2-beeb2e544198 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:45.896521 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae4949-d4b9-4f9d-ae85-71edb94f2ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:45.897176 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae4949-d4b9-4f9d-ae85-71edb94f2ddf admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:45.897465 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae4949-d4b9-4f9d-ae85-71edb94f2ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:45.897746 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae4949-d4b9-4f9d-ae85-71edb94f2ddf admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:45.898460 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae4949-d4b9-4f9d-ae85-71edb94f2ddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:45.900333 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae4949-d4b9-4f9d-ae85-71edb94f2ddf admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:45.910588 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1253/2483] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:27:45.914870 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c789660-4183-473b-afd1-471e33170aec admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:45.925555 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c789660-4183-473b-afd1-471e33170aec admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:45.930145 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8cfe0f-8fc1-4bf2-b2c2-7744e75235b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:45.931016 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8cfe0f-8fc1-4bf2-b2c2-7744e75235b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:45.931316 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8cfe0f-8fc1-4bf2-b2c2-7744e75235b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:45.931612 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8cfe0f-8fc1-4bf2-b2c2-7744e75235b6 admin admin] PATH_INFO: `/v3/projects/b750fbe5306a4484bd3305004749fdf6/users/52218b6d68d4464f8c63f748c4c5255a/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:45.933608 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8cfe0f-8fc1-4bf2-b2c2-7744e75235b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b750fbe5306a4484bd3305004749fdf6, user_id=52218b6d68d4464f8c63f748c4c5255a, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:45.936349 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8cfe0f-8fc1-4bf2-b2c2-7744e75235b6 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:45.946247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1231/2484] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:27:45 2022] PUT /identity/v3/projects/b750fbe5306a4484bd3305004749fdf6/users/52218b6d68d4464f8c63f748c4c5255a/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:27:45.957414 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84b942-3589-4722-8b3f-35323bf8c834 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:45.957715 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84b942-3589-4722-8b3f-35323bf8c834 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:45.957999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84b942-3589-4722-8b3f-35323bf8c834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:45.985111 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-2c84b942-3589-4722-8b3f-35323bf8c834 None None] MFA Rules not processed for user `52218b6d68d4464f8c63f748c4c5255a`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:27:45.986426 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-2c84b942-3589-4722-8b3f-35323bf8c834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:27:46.053016 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1254/2485] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:27:45 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:27:46.064217 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8cfe0f-8fc1-4bf2-b2c2-7744e75235b6 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:46.065943 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2a8cfe0f-8fc1-4bf2-b2c2-7744e75235b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:27:46.075757 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:27:46.129233 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8cfe0f-8fc1-4bf2-b2c2-7744e75235b6 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:46.133038 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd32a88-67f4-4ed1-8e12-1bbb826f421d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:46.133681 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd32a88-67f4-4ed1-8e12-1bbb826f421d service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:46.134097 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd32a88-67f4-4ed1-8e12-1bbb826f421d service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:46.134384 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd32a88-67f4-4ed1-8e12-1bbb826f421d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:46.143797 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd32a88-67f4-4ed1-8e12-1bbb826f421d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:46.145364 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd32a88-67f4-4ed1-8e12-1bbb826f421d service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:46.154846 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1232/2486] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:27:46 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:27:46.224636 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c84b942-3589-4722-8b3f-35323bf8c834 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:27:46.234772 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c84b942-3589-4722-8b3f-35323bf8c834 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:27:46.238396 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ec3aa-20a8-401f-ba8e-2044bd249361 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:27:46.239036 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ec3aa-20a8-401f-ba8e-2044bd249361 service cinder] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:27:46.239351 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ec3aa-20a8-401f-ba8e-2044bd249361 service cinder] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:27:46.239634 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ec3aa-20a8-401f-ba8e-2044bd249361 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:27:46.258090 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:27:46.301504 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1ec3aa-20a8-401f-ba8e-2044bd249361 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:27:46.302981 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1ec3aa-20a8-401f-ba8e-2044bd249361 service cinder] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:27:46.311922 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1255/2487] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:27:46 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:27:56.150786 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=23 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:27:56.307773 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=19 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:01.141480 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd32a88-67f4-4ed1-8e12-1bbb826f421d service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:01.262970 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd32a88-67f4-4ed1-8e12-1bbb826f421d service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:01.268394 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4638b490-89e6-4ba3-a244-f58e6347b2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:01.269289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4638b490-89e6-4ba3-a244-f58e6347b2f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:01.269575 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4638b490-89e6-4ba3-a244-f58e6347b2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:01.269851 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4638b490-89e6-4ba3-a244-f58e6347b2f7 admin admin] PATH_INFO: `/v3/users/6e1074fa7fce448bb412ed47270344b3` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:01.271133 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4638b490-89e6-4ba3-a244-f58e6347b2f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e1074fa7fce448bb412ed47270344b3)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:01.273126 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4638b490-89e6-4ba3-a244-f58e6347b2f7 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:01.321778 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4638b490-89e6-4ba3-a244-f58e6347b2f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e1074fa7fce448bb412ed47270344b3'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:01.329354 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4638b490-89e6-4ba3-a244-f58e6347b2f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e1074fa7fce448bb412ed47270344b3'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:01.340942 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4638b490-89e6-4ba3-a244-f58e6347b2f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e1074fa7fce448bb412ed47270344b3'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:01.341553 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4638b490-89e6-4ba3-a244-f58e6347b2f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e1074fa7fce448bb412ed47270344b3'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:01.348799 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4638b490-89e6-4ba3-a244-f58e6347b2f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e1074fa7fce448bb412ed47270344b3'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:01.354527 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1233/2488] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:28:01 2022] DELETE /identity/v3/users/6e1074fa7fce448bb412ed47270344b3 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:01.359046 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30eca9-f23f-4bb1-abba-38d9f23789a3 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:01.359547 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30eca9-f23f-4bb1-abba-38d9f23789a3 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:01.359659 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30eca9-f23f-4bb1-abba-38d9f23789a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:01.406535 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-0a30eca9-f23f-4bb1-abba-38d9f23789a3 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:01.407784 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-0a30eca9-f23f-4bb1-abba-38d9f23789a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:01.464210 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1256/2489] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:01.473877 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4638b490-89e6-4ba3-a244-f58e6347b2f7 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:01.475320 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4638b490-89e6-4ba3-a244-f58e6347b2f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:01.517379 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:01.550936 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4638b490-89e6-4ba3-a244-f58e6347b2f7 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:01.554661 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c18910-3206-4dab-aa00-a131fae8f737 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:01.559210 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c18910-3206-4dab-aa00-a131fae8f737 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:01.559210 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c18910-3206-4dab-aa00-a131fae8f737 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:01.559210 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c18910-3206-4dab-aa00-a131fae8f737 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:01.565578 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c18910-3206-4dab-aa00-a131fae8f737 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:01.567015 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c18910-3206-4dab-aa00-a131fae8f737 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:01.576396 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1234/2490] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:28:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 16:28:01.843846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30eca9-f23f-4bb1-abba-38d9f23789a3 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:01.845399 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-0a30eca9-f23f-4bb1-abba-38d9f23789a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:01.860239 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30eca9-f23f-4bb1-abba-38d9f23789a3 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:01.865689 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38c636-cd1e-4c05-becb-2d31acf1fd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:01.866583 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38c636-cd1e-4c05-becb-2d31acf1fd54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:01.866861 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38c636-cd1e-4c05-becb-2d31acf1fd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:01.867242 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38c636-cd1e-4c05-becb-2d31acf1fd54 admin admin] PATH_INFO: `/v3/projects/3a764b1e21f843c7a8680fc5eed1d6dc` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:01.868362 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d38c636-cd1e-4c05-becb-2d31acf1fd54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a764b1e21f843c7a8680fc5eed1d6dc)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:01.870291 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d38c636-cd1e-4c05-becb-2d31acf1fd54 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:01.943639 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-1d38c636-cd1e-4c05-becb-2d31acf1fd54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a764b1e21f843c7a8680fc5eed1d6dc', 'request_id': 'req-1d38c636-cd1e-4c05-becb-2d31acf1fd54'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:01.949688 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-1d38c636-cd1e-4c05-becb-2d31acf1fd54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a764b1e21f843c7a8680fc5eed1d6dc', 'request_id': 'req-1d38c636-cd1e-4c05-becb-2d31acf1fd54'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:01.957428 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-1d38c636-cd1e-4c05-becb-2d31acf1fd54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a764b1e21f843c7a8680fc5eed1d6dc', 'request_id': 'req-1d38c636-cd1e-4c05-becb-2d31acf1fd54'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:01.967306 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-1d38c636-cd1e-4c05-becb-2d31acf1fd54 admin admin] The token cache is being invalidate because project 3a764b1e21f843c7a8680fc5eed1d6dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:28:01.967612 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-1d38c636-cd1e-4c05-becb-2d31acf1fd54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:01.968756 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-1d38c636-cd1e-4c05-becb-2d31acf1fd54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:01.970293 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1257/2491] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:28:01 2022] DELETE /identity/v3/projects/3a764b1e21f843c7a8680fc5eed1d6dc => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:01.974263 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c18910-3206-4dab-aa00-a131fae8f737 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:01.984008 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c18910-3206-4dab-aa00-a131fae8f737 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:02.039100 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7248d80-da08-4e54-950b-58c66856c967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:02.052160 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7248d80-da08-4e54-950b-58c66856c967 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:02.052160 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7248d80-da08-4e54-950b-58c66856c967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:02.052160 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7248d80-da08-4e54-950b-58c66856c967 admin admin] PATH_INFO: `/v3/users/52218b6d68d4464f8c63f748c4c5255a` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:02.052160 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7248d80-da08-4e54-950b-58c66856c967 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52218b6d68d4464f8c63f748c4c5255a)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:02.052160 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7248d80-da08-4e54-950b-58c66856c967 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:02.093663 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d7248d80-da08-4e54-950b-58c66856c967 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52218b6d68d4464f8c63f748c4c5255a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:02.100189 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d7248d80-da08-4e54-950b-58c66856c967 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52218b6d68d4464f8c63f748c4c5255a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:02.111917 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d7248d80-da08-4e54-950b-58c66856c967 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52218b6d68d4464f8c63f748c4c5255a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:02.112554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d7248d80-da08-4e54-950b-58c66856c967 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52218b6d68d4464f8c63f748c4c5255a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:02.119540 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d7248d80-da08-4e54-950b-58c66856c967 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52218b6d68d4464f8c63f748c4c5255a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:02.126025 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1235/2492] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:28:01 2022] DELETE /identity/v3/users/52218b6d68d4464f8c63f748c4c5255a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:02.387920 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38c636-cd1e-4c05-becb-2d31acf1fd54 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:02.390408 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-1d38c636-cd1e-4c05-becb-2d31acf1fd54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:02.411637 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38c636-cd1e-4c05-becb-2d31acf1fd54 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:02.462162 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:02.469458 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc351def-48ca-459b-94a3-ee68fe58bdcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:02.470676 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc351def-48ca-459b-94a3-ee68fe58bdcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:02.471073 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc351def-48ca-459b-94a3-ee68fe58bdcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:02.471597 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc351def-48ca-459b-94a3-ee68fe58bdcf admin admin] PATH_INFO: `/v3/projects/b750fbe5306a4484bd3305004749fdf6` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:02.472967 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc351def-48ca-459b-94a3-ee68fe58bdcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=b750fbe5306a4484bd3305004749fdf6)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:02.474841 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc351def-48ca-459b-94a3-ee68fe58bdcf admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:02.520245 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-cc351def-48ca-459b-94a3-ee68fe58bdcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b750fbe5306a4484bd3305004749fdf6', 'request_id': 'req-cc351def-48ca-459b-94a3-ee68fe58bdcf'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:02.526635 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-cc351def-48ca-459b-94a3-ee68fe58bdcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b750fbe5306a4484bd3305004749fdf6', 'request_id': 'req-cc351def-48ca-459b-94a3-ee68fe58bdcf'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:02.534337 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-cc351def-48ca-459b-94a3-ee68fe58bdcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b750fbe5306a4484bd3305004749fdf6', 'request_id': 'req-cc351def-48ca-459b-94a3-ee68fe58bdcf'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:02.545281 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-cc351def-48ca-459b-94a3-ee68fe58bdcf admin admin] The token cache is being invalidate because project b750fbe5306a4484bd3305004749fdf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:28:02.545594 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-cc351def-48ca-459b-94a3-ee68fe58bdcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:02.546273 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-cc351def-48ca-459b-94a3-ee68fe58bdcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:02.547954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1258/2493] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:28:02 2022] DELETE /identity/v3/projects/b750fbe5306a4484bd3305004749fdf6 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:28:02.553044 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7ab19-b6a5-49e0-998a-4fcbbac3a66b None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:02.553518 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7ab19-b6a5-49e0-998a-4fcbbac3a66b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:02.553645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7ab19-b6a5-49e0-998a-4fcbbac3a66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:02.592071 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-39e7ab19-b6a5-49e0-998a-4fcbbac3a66b None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:02.593324 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-39e7ab19-b6a5-49e0-998a-4fcbbac3a66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:02.617093 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:02.653463 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1236/2494] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:02.661224 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f49df0-dfa3-4dbe-a232-9a978236368d None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:02.661515 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f49df0-dfa3-4dbe-a232-9a978236368d None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:02.661791 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f49df0-dfa3-4dbe-a232-9a978236368d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:02.680135 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-20f49df0-dfa3-4dbe-a232-9a978236368d None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:02.681318 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-20f49df0-dfa3-4dbe-a232-9a978236368d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:02.689087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1259/2495] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:02.693125 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e7ab19-b6a5-49e0-998a-4fcbbac3a66b None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:02.703080 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e7ab19-b6a5-49e0-998a-4fcbbac3a66b None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:02.707671 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65abf10e-67fe-4a16-9ed1-c9495a212dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:02.708338 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65abf10e-67fe-4a16-9ed1-c9495a212dc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:02.708626 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65abf10e-67fe-4a16-9ed1-c9495a212dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:02.708904 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65abf10e-67fe-4a16-9ed1-c9495a212dc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:02.709634 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65abf10e-67fe-4a16-9ed1-c9495a212dc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:28:02.709947 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65abf10e-67fe-4a16-9ed1-c9495a212dc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:02.711571 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65abf10e-67fe-4a16-9ed1-c9495a212dc8 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:02.723379 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1237/2496] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:28:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:28:02.727493 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f49df0-dfa3-4dbe-a232-9a978236368d None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:02.736459 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f49df0-dfa3-4dbe-a232-9a978236368d None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:02.741038 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b742482-56c8-4b30-8b9c-9fc58d7a38c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:02.741703 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b742482-56c8-4b30-8b9c-9fc58d7a38c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:02.741985 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b742482-56c8-4b30-8b9c-9fc58d7a38c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:02.742261 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b742482-56c8-4b30-8b9c-9fc58d7a38c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:02.742968 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b742482-56c8-4b30-8b9c-9fc58d7a38c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:02.744880 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b742482-56c8-4b30-8b9c-9fc58d7a38c3 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:02.769394 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1260/2497] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:28:02 2022] POST /identity/v3/projects => generated 385 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:28:02.773371 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65abf10e-67fe-4a16-9ed1-c9495a212dc8 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:02.783587 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65abf10e-67fe-4a16-9ed1-c9495a212dc8 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:02.837676 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c14137-cdaf-475a-b044-ffc4eb786488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:02.838333 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c14137-cdaf-475a-b044-ffc4eb786488 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:02.838618 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c14137-cdaf-475a-b044-ffc4eb786488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:02.838899 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c14137-cdaf-475a-b044-ffc4eb786488 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:02.839624 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c14137-cdaf-475a-b044-ffc4eb786488 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:02.841545 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c14137-cdaf-475a-b044-ffc4eb786488 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:02.862114 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1238/2498] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:28:02 2022] POST /identity/v3/users => generated 293 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:28:02.866222 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b742482-56c8-4b30-8b9c-9fc58d7a38c3 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:02.876451 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b742482-56c8-4b30-8b9c-9fc58d7a38c3 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:02.881016 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e1bc3-ff24-44da-967a-a0c747228f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:02.881660 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e1bc3-ff24-44da-967a-a0c747228f0e admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:02.881940 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e1bc3-ff24-44da-967a-a0c747228f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:02.882244 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e1bc3-ff24-44da-967a-a0c747228f0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:02.882962 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e1bc3-ff24-44da-967a-a0c747228f0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:02.884573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e1bc3-ff24-44da-967a-a0c747228f0e admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:02.895075 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1261/2499] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:28:02.898789 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c14137-cdaf-475a-b044-ffc4eb786488 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:02.909546 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c14137-cdaf-475a-b044-ffc4eb786488 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:02.914216 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6935ea15-b0f0-47a4-800d-55783a8b4c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:02.914868 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935ea15-b0f0-47a4-800d-55783a8b4c4f admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:02.915157 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935ea15-b0f0-47a4-800d-55783a8b4c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:02.915465 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935ea15-b0f0-47a4-800d-55783a8b4c4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:02.916453 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6935ea15-b0f0-47a4-800d-55783a8b4c4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:02.917953 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6935ea15-b0f0-47a4-800d-55783a8b4c4f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:02.929812 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1239/2500] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:28:02.934044 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e1bc3-ff24-44da-967a-a0c747228f0e admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:02.944132 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e1bc3-ff24-44da-967a-a0c747228f0e admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:02.948719 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a7edde-16da-4d76-a53c-bda0487943d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:02.949592 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7edde-16da-4d76-a53c-bda0487943d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:02.949871 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7edde-16da-4d76-a53c-bda0487943d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:02.950149 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7edde-16da-4d76-a53c-bda0487943d3 admin admin] PATH_INFO: `/v3/projects/07b751ee757f4c928963dc7ecd5325fe/users/ce78ea56ec9b44e385e49e8ee65e0019/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:02.965441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a7edde-16da-4d76-a53c-bda0487943d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07b751ee757f4c928963dc7ecd5325fe, user_id=ce78ea56ec9b44e385e49e8ee65e0019, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:02.969154 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a7edde-16da-4d76-a53c-bda0487943d3 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:02.982565 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1262/2501] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:28:02 2022] PUT /identity/v3/projects/07b751ee757f4c928963dc7ecd5325fe/users/ce78ea56ec9b44e385e49e8ee65e0019/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:02.993805 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa22ce2-dac8-4c77-9428-8a138b1d6fa0 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:02.994104 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa22ce2-dac8-4c77-9428-8a138b1d6fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:02.994470 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa22ce2-dac8-4c77-9428-8a138b1d6fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:03.022011 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-8aa22ce2-dac8-4c77-9428-8a138b1d6fa0 None None] MFA Rules not processed for user `ce78ea56ec9b44e385e49e8ee65e0019`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:03.023236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8aa22ce2-dac8-4c77-9428-8a138b1d6fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:03.089744 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1240/2502] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:03.095017 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a7edde-16da-4d76-a53c-bda0487943d3 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:03.166188 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a7edde-16da-4d76-a53c-bda0487943d3 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:03.170970 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e17b5-0e05-49aa-bea6-e1a53127c75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:03.171810 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e17b5-0e05-49aa-bea6-e1a53127c75b admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:03.172101 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e17b5-0e05-49aa-bea6-e1a53127c75b admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:03.172371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e17b5-0e05-49aa-bea6-e1a53127c75b admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:03.173047 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730e17b5-0e05-49aa-bea6-e1a53127c75b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:03.174926 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730e17b5-0e05-49aa-bea6-e1a53127c75b admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:03.206286 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1263/2503] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:28:03 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:28:03.210318 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa22ce2-dac8-4c77-9428-8a138b1d6fa0 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:03.221424 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa22ce2-dac8-4c77-9428-8a138b1d6fa0 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:03.244452 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:03.287205 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe9139b-dff8-4a9f-a054-766d5954b2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:03.288051 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9139b-dff8-4a9f-a054-766d5954b2d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:03.288341 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9139b-dff8-4a9f-a054-766d5954b2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:03.288672 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9139b-dff8-4a9f-a054-766d5954b2d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:03.289364 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe9139b-dff8-4a9f-a054-766d5954b2d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:03.291372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe9139b-dff8-4a9f-a054-766d5954b2d4 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:03.312687 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1241/2504] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:28:03 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:28:03.316710 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e17b5-0e05-49aa-bea6-e1a53127c75b admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:03.326798 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e17b5-0e05-49aa-bea6-e1a53127c75b admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:03.331985 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d2eabd-8a60-495c-b666-536b5c8193ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:03.332638 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2eabd-8a60-495c-b666-536b5c8193ac admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:03.332918 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2eabd-8a60-495c-b666-536b5c8193ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:03.333193 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2eabd-8a60-495c-b666-536b5c8193ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:03.333935 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d2eabd-8a60-495c-b666-536b5c8193ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:03.335533 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d2eabd-8a60-495c-b666-536b5c8193ac admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:03.337527 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:03.347874 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1264/2505] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:28:03.351636 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe9139b-dff8-4a9f-a054-766d5954b2d4 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:03.361453 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe9139b-dff8-4a9f-a054-766d5954b2d4 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:03.366049 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4a71d-150e-4490-88da-774cdb32af14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:03.366699 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4a71d-150e-4490-88da-774cdb32af14 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:03.366989 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4a71d-150e-4490-88da-774cdb32af14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:03.367270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4a71d-150e-4490-88da-774cdb32af14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:03.368055 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba4a71d-150e-4490-88da-774cdb32af14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:03.369659 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba4a71d-150e-4490-88da-774cdb32af14 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:03.380057 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1242/2506] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:28:03.384164 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d2eabd-8a60-495c-b666-536b5c8193ac admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:03.394623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d2eabd-8a60-495c-b666-536b5c8193ac admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:03.399127 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f81f61-4f17-4b57-8dee-fb06d6bfa38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:03.400321 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f81f61-4f17-4b57-8dee-fb06d6bfa38b admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:03.400603 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f81f61-4f17-4b57-8dee-fb06d6bfa38b admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:03.400876 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f81f61-4f17-4b57-8dee-fb06d6bfa38b admin admin] PATH_INFO: `/v3/projects/3451ab6045114e54b2ff5ed3502441b4/users/5e8ea60c8b5843eeb6b2ed230d6feaa9/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:03.414837 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f81f61-4f17-4b57-8dee-fb06d6bfa38b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3451ab6045114e54b2ff5ed3502441b4, user_id=5e8ea60c8b5843eeb6b2ed230d6feaa9, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:03.417508 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f81f61-4f17-4b57-8dee-fb06d6bfa38b admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:03.430457 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1265/2507] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:28:03 2022] PUT /identity/v3/projects/3451ab6045114e54b2ff5ed3502441b4/users/5e8ea60c8b5843eeb6b2ed230d6feaa9/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:28:03.441049 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e674c-c941-4838-b066-38e72b7da479 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:03.441347 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e674c-c941-4838-b066-38e72b7da479 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:03.441632 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e674c-c941-4838-b066-38e72b7da479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:03.467003 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-0d6e674c-c941-4838-b066-38e72b7da479 None None] MFA Rules not processed for user `5e8ea60c8b5843eeb6b2ed230d6feaa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:03.468257 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0d6e674c-c941-4838-b066-38e72b7da479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:03.530514 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1243/2508] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:03.534934 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f81f61-4f17-4b57-8dee-fb06d6bfa38b admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:03.595305 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f81f61-4f17-4b57-8dee-fb06d6bfa38b admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:03.599938 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9420e6d5-5f59-41c2-ba22-cc3ddb839ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:03.600588 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420e6d5-5f59-41c2-ba22-cc3ddb839ea7 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:03.600875 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420e6d5-5f59-41c2-ba22-cc3ddb839ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:03.601152 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420e6d5-5f59-41c2-ba22-cc3ddb839ea7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:03.601836 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9420e6d5-5f59-41c2-ba22-cc3ddb839ea7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:03.603771 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9420e6d5-5f59-41c2-ba22-cc3ddb839ea7 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:03.629145 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1266/2509] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:28:03 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:28:03.633170 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e674c-c941-4838-b066-38e72b7da479 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:03.643862 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e674c-c941-4838-b066-38e72b7da479 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:03.699083 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879decc9-97e1-47be-8f1c-fd3aabd09528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:03.699795 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879decc9-97e1-47be-8f1c-fd3aabd09528 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:03.700078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879decc9-97e1-47be-8f1c-fd3aabd09528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:03.700355 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879decc9-97e1-47be-8f1c-fd3aabd09528 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:03.701033 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879decc9-97e1-47be-8f1c-fd3aabd09528 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:03.702992 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879decc9-97e1-47be-8f1c-fd3aabd09528 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:03.722285 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1244/2510] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:28:03 2022] POST /identity/v3/users => generated 299 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:28:03.726217 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9420e6d5-5f59-41c2-ba22-cc3ddb839ea7 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:03.736851 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9420e6d5-5f59-41c2-ba22-cc3ddb839ea7 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:03.741488 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7d53e-6e42-4cc8-85ee-b6f70d819c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:03.742081 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7d53e-6e42-4cc8-85ee-b6f70d819c21 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:03.742365 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7d53e-6e42-4cc8-85ee-b6f70d819c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:03.742642 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7d53e-6e42-4cc8-85ee-b6f70d819c21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:03.743381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7d53e-6e42-4cc8-85ee-b6f70d819c21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:03.744952 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7d53e-6e42-4cc8-85ee-b6f70d819c21 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:03.755720 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1267/2511] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:28:03.759549 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879decc9-97e1-47be-8f1c-fd3aabd09528 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:03.769330 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879decc9-97e1-47be-8f1c-fd3aabd09528 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:03.773960 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4505146-ae14-495a-bb0b-17ad62143533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:03.774828 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4505146-ae14-495a-bb0b-17ad62143533 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:03.775419 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4505146-ae14-495a-bb0b-17ad62143533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:03.775699 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4505146-ae14-495a-bb0b-17ad62143533 admin admin] PATH_INFO: `/v3/projects/ac77e64a12d9491b9233671cc5b52a28/users/e34cbed578fd4fe2828f924cd9e73c8e/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:03.791123 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4505146-ae14-495a-bb0b-17ad62143533 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac77e64a12d9491b9233671cc5b52a28, user_id=e34cbed578fd4fe2828f924cd9e73c8e, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:03.794000 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4505146-ae14-495a-bb0b-17ad62143533 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:03.801947 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:03.808671 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1245/2512] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:28:03 2022] PUT /identity/v3/projects/ac77e64a12d9491b9233671cc5b52a28/users/e34cbed578fd4fe2828f924cd9e73c8e/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:03.812727 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7d53e-6e42-4cc8-85ee-b6f70d819c21 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:03.822426 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7d53e-6e42-4cc8-85ee-b6f70d819c21 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:03.877929 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a3f7c-6b03-4416-8697-c482808b9691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:03.878635 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a3f7c-6b03-4416-8697-c482808b9691 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:03.878933 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a3f7c-6b03-4416-8697-c482808b9691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:03.879238 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a3f7c-6b03-4416-8697-c482808b9691 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:03.880121 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a3f7c-6b03-4416-8697-c482808b9691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:03.881770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a3f7c-6b03-4416-8697-c482808b9691 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:03.891415 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1268/2513] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:28:03.895462 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4505146-ae14-495a-bb0b-17ad62143533 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:03.906137 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4505146-ae14-495a-bb0b-17ad62143533 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:03.910733 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076eece7-8309-4d50-8924-b67457a3384c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:03.911633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076eece7-8309-4d50-8924-b67457a3384c admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:03.911917 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076eece7-8309-4d50-8924-b67457a3384c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:03.912193 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076eece7-8309-4d50-8924-b67457a3384c admin admin] PATH_INFO: `/v3/projects/ac77e64a12d9491b9233671cc5b52a28/users/e34cbed578fd4fe2828f924cd9e73c8e/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:03.914132 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076eece7-8309-4d50-8924-b67457a3384c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac77e64a12d9491b9233671cc5b52a28, user_id=e34cbed578fd4fe2828f924cd9e73c8e, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:03.916789 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076eece7-8309-4d50-8924-b67457a3384c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:03.928136 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1246/2514] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:28:03 2022] PUT /identity/v3/projects/ac77e64a12d9491b9233671cc5b52a28/users/e34cbed578fd4fe2828f924cd9e73c8e/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:03.937953 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779da3ef-b499-4d18-bc27-f2b3befd1aa2 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:03.938243 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779da3ef-b499-4d18-bc27-f2b3befd1aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:03.938659 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779da3ef-b499-4d18-bc27-f2b3befd1aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:03.968530 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-779da3ef-b499-4d18-bc27-f2b3befd1aa2 None None] MFA Rules not processed for user `e34cbed578fd4fe2828f924cd9e73c8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:03.969737 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-779da3ef-b499-4d18-bc27-f2b3befd1aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:04.017408 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:04.056123 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1269/2515] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:03 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:04.067168 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076eece7-8309-4d50-8924-b67457a3384c admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:04.068883 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-076eece7-8309-4d50-8924-b67457a3384c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:04.125256 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076eece7-8309-4d50-8924-b67457a3384c admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:04.128931 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9bc89-98b6-4452-b256-99069ba4b085 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:04.129568 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9bc89-98b6-4452-b256-99069ba4b085 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:04.129858 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9bc89-98b6-4452-b256-99069ba4b085 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:04.130136 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9bc89-98b6-4452-b256-99069ba4b085 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:04.138917 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad9bc89-98b6-4452-b256-99069ba4b085 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:04.140868 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad9bc89-98b6-4452-b256-99069ba4b085 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:04.150378 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1247/2516] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:28:04 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:28:04.669162 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779da3ef-b499-4d18-bc27-f2b3befd1aa2 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:04.679214 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779da3ef-b499-4d18-bc27-f2b3befd1aa2 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:04.682967 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e416a4-8f58-4428-a8f1-9ffc83e16488 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:04.683762 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e416a4-8f58-4428-a8f1-9ffc83e16488 service cinder] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:04.684075 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e416a4-8f58-4428-a8f1-9ffc83e16488 service cinder] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:04.684380 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e416a4-8f58-4428-a8f1-9ffc83e16488 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:04.746526 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e416a4-8f58-4428-a8f1-9ffc83e16488 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:04.748004 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e416a4-8f58-4428-a8f1-9ffc83e16488 service cinder] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:04.757278 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1270/2517] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:28:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:28:08.715720 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9bc89-98b6-4452-b256-99069ba4b085 service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:08.717129 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8ad9bc89-98b6-4452-b256-99069ba4b085 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:08.728456 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9bc89-98b6-4452-b256-99069ba4b085 service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:08.732171 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc0bb1-8415-41e6-a249-173d853ee2a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:08.732810 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc0bb1-8415-41e6-a249-173d853ee2a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:08.733097 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc0bb1-8415-41e6-a249-173d853ee2a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:08.733374 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc0bb1-8415-41e6-a249-173d853ee2a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:08.788581 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dc0bb1-8415-41e6-a249-173d853ee2a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:08.789999 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dc0bb1-8415-41e6-a249-173d853ee2a6 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:08.799370 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1248/2518] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:28:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:28:10.831026 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e416a4-8f58-4428-a8f1-9ffc83e16488 service cinder] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:10.892920 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e416a4-8f58-4428-a8f1-9ffc83e16488 service cinder] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:10.897186 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa3ced6-05ff-4b0a-b6a1-4317b0bb5682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:10.898054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3ced6-05ff-4b0a-b6a1-4317b0bb5682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:10.898453 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3ced6-05ff-4b0a-b6a1-4317b0bb5682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:10.898602 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3ced6-05ff-4b0a-b6a1-4317b0bb5682 admin admin] PATH_INFO: `/v3/users/ce78ea56ec9b44e385e49e8ee65e0019` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:10.899795 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa3ced6-05ff-4b0a-b6a1-4317b0bb5682 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce78ea56ec9b44e385e49e8ee65e0019)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:10.901964 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa3ced6-05ff-4b0a-b6a1-4317b0bb5682 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:10.948686 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-cfa3ced6-05ff-4b0a-b6a1-4317b0bb5682 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce78ea56ec9b44e385e49e8ee65e0019'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:10.956572 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-cfa3ced6-05ff-4b0a-b6a1-4317b0bb5682 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce78ea56ec9b44e385e49e8ee65e0019'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:10.969395 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:10.970744 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-cfa3ced6-05ff-4b0a-b6a1-4317b0bb5682 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce78ea56ec9b44e385e49e8ee65e0019'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:10.979146 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-cfa3ced6-05ff-4b0a-b6a1-4317b0bb5682 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce78ea56ec9b44e385e49e8ee65e0019'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:10.979869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-cfa3ced6-05ff-4b0a-b6a1-4317b0bb5682 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce78ea56ec9b44e385e49e8ee65e0019'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:10.985721 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1271/2519] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:28:10 2022] DELETE /identity/v3/users/ce78ea56ec9b44e385e49e8ee65e0019 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:10.990464 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a31b33-9956-4171-9fef-eb3aa552c5c8 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:10.990944 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a31b33-9956-4171-9fef-eb3aa552c5c8 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:10.991062 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a31b33-9956-4171-9fef-eb3aa552c5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:11.008259 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-17a31b33-9956-4171-9fef-eb3aa552c5c8 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:11.009470 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-17a31b33-9956-4171-9fef-eb3aa552c5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:11.067824 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1249/2520] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:11.077927 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa3ced6-05ff-4b0a-b6a1-4317b0bb5682 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:11.079352 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-cfa3ced6-05ff-4b0a-b6a1-4317b0bb5682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:11.157555 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa3ced6-05ff-4b0a-b6a1-4317b0bb5682 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:11.161261 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8b751-14c9-4b82-b186-317fa51cab4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:11.161906 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8b751-14c9-4b82-b186-317fa51cab4e service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:11.162191 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8b751-14c9-4b82-b186-317fa51cab4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:11.162469 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8b751-14c9-4b82-b186-317fa51cab4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:11.172266 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8b751-14c9-4b82-b186-317fa51cab4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:11.173758 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8b751-14c9-4b82-b186-317fa51cab4e service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:11.185442 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1272/2521] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:28:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:28:11.472153 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a31b33-9956-4171-9fef-eb3aa552c5c8 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:11.473593 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-17a31b33-9956-4171-9fef-eb3aa552c5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:11.485755 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a31b33-9956-4171-9fef-eb3aa552c5c8 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:11.491025 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d5fdc7-5f86-426f-bc9a-862dec094a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:11.492293 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5fdc7-5f86-426f-bc9a-862dec094a23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:11.492645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5fdc7-5f86-426f-bc9a-862dec094a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:11.493103 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5fdc7-5f86-426f-bc9a-862dec094a23 admin admin] PATH_INFO: `/v3/projects/07b751ee757f4c928963dc7ecd5325fe` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:11.494392 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d5fdc7-5f86-426f-bc9a-862dec094a23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07b751ee757f4c928963dc7ecd5325fe)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:11.496454 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d5fdc7-5f86-426f-bc9a-862dec094a23 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:11.521718 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:11.544639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-53d5fdc7-5f86-426f-bc9a-862dec094a23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07b751ee757f4c928963dc7ecd5325fe', 'request_id': 'req-53d5fdc7-5f86-426f-bc9a-862dec094a23'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:11.550595 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-53d5fdc7-5f86-426f-bc9a-862dec094a23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07b751ee757f4c928963dc7ecd5325fe', 'request_id': 'req-53d5fdc7-5f86-426f-bc9a-862dec094a23'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:11.557553 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-53d5fdc7-5f86-426f-bc9a-862dec094a23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07b751ee757f4c928963dc7ecd5325fe', 'request_id': 'req-53d5fdc7-5f86-426f-bc9a-862dec094a23'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:11.568025 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-53d5fdc7-5f86-426f-bc9a-862dec094a23 admin admin] The token cache is being invalidate because project 07b751ee757f4c928963dc7ecd5325fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:28:11.568311 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-53d5fdc7-5f86-426f-bc9a-862dec094a23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:11.568921 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-53d5fdc7-5f86-426f-bc9a-862dec094a23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:11.570434 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1250/2522] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:28:11 2022] DELETE /identity/v3/projects/07b751ee757f4c928963dc7ecd5325fe => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:11.574449 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8b751-14c9-4b82-b186-317fa51cab4e service neutron] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:11.584824 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8b751-14c9-4b82-b186-317fa51cab4e service neutron] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:11.639541 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369aad41-e5ea-430f-a676-410688dc2e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:11.640961 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369aad41-e5ea-430f-a676-410688dc2e1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:11.641243 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369aad41-e5ea-430f-a676-410688dc2e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:11.641515 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369aad41-e5ea-430f-a676-410688dc2e1f admin admin] PATH_INFO: `/v3/users/5e8ea60c8b5843eeb6b2ed230d6feaa9` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:11.642672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369aad41-e5ea-430f-a676-410688dc2e1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e8ea60c8b5843eeb6b2ed230d6feaa9)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:11.644623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369aad41-e5ea-430f-a676-410688dc2e1f admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:11.689151 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-369aad41-e5ea-430f-a676-410688dc2e1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e8ea60c8b5843eeb6b2ed230d6feaa9'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:11.696606 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-369aad41-e5ea-430f-a676-410688dc2e1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e8ea60c8b5843eeb6b2ed230d6feaa9'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:11.708374 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-369aad41-e5ea-430f-a676-410688dc2e1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e8ea60c8b5843eeb6b2ed230d6feaa9'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:11.709527 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:11.716646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-369aad41-e5ea-430f-a676-410688dc2e1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e8ea60c8b5843eeb6b2ed230d6feaa9'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:11.717563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-369aad41-e5ea-430f-a676-410688dc2e1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e8ea60c8b5843eeb6b2ed230d6feaa9'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:11.723380 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1273/2523] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:28:11 2022] DELETE /identity/v3/users/5e8ea60c8b5843eeb6b2ed230d6feaa9 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:28:11.992333 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d5fdc7-5f86-426f-bc9a-862dec094a23 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:11.994340 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-53d5fdc7-5f86-426f-bc9a-862dec094a23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:12.020952 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d5fdc7-5f86-426f-bc9a-862dec094a23 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:12.079983 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360c211-44be-4c1f-994f-8d83ef0f412b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:12.080843 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360c211-44be-4c1f-994f-8d83ef0f412b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:12.081128 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360c211-44be-4c1f-994f-8d83ef0f412b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:12.081408 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360c211-44be-4c1f-994f-8d83ef0f412b admin admin] PATH_INFO: `/v3/projects/3451ab6045114e54b2ff5ed3502441b4` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:12.082600 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a360c211-44be-4c1f-994f-8d83ef0f412b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3451ab6045114e54b2ff5ed3502441b4)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:12.084560 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a360c211-44be-4c1f-994f-8d83ef0f412b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:12.133576 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a360c211-44be-4c1f-994f-8d83ef0f412b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3451ab6045114e54b2ff5ed3502441b4', 'request_id': 'req-a360c211-44be-4c1f-994f-8d83ef0f412b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:12.140922 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a360c211-44be-4c1f-994f-8d83ef0f412b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3451ab6045114e54b2ff5ed3502441b4', 'request_id': 'req-a360c211-44be-4c1f-994f-8d83ef0f412b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:12.148631 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a360c211-44be-4c1f-994f-8d83ef0f412b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3451ab6045114e54b2ff5ed3502441b4', 'request_id': 'req-a360c211-44be-4c1f-994f-8d83ef0f412b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:12.159926 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a360c211-44be-4c1f-994f-8d83ef0f412b admin admin] The token cache is being invalidate because project 3451ab6045114e54b2ff5ed3502441b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:28:12.160219 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a360c211-44be-4c1f-994f-8d83ef0f412b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:12.160864 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a360c211-44be-4c1f-994f-8d83ef0f412b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:12.162399 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1251/2524] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:28:11 2022] DELETE /identity/v3/projects/3451ab6045114e54b2ff5ed3502441b4 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:12.166171 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369aad41-e5ea-430f-a676-410688dc2e1f admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:12.167705 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-369aad41-e5ea-430f-a676-410688dc2e1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:12.180235 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369aad41-e5ea-430f-a676-410688dc2e1f admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:12.243320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8374cd-fc61-44af-b9ab-f3294322e666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:12.244320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8374cd-fc61-44af-b9ab-f3294322e666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:12.244541 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8374cd-fc61-44af-b9ab-f3294322e666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:12.244840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8374cd-fc61-44af-b9ab-f3294322e666 admin admin] PATH_INFO: `/v3/users/e34cbed578fd4fe2828f924cd9e73c8e` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:12.246042 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8374cd-fc61-44af-b9ab-f3294322e666 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e34cbed578fd4fe2828f924cd9e73c8e)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:12.248159 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8374cd-fc61-44af-b9ab-f3294322e666 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:12.295468 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-9c8374cd-fc61-44af-b9ab-f3294322e666 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e34cbed578fd4fe2828f924cd9e73c8e'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:12.302928 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-9c8374cd-fc61-44af-b9ab-f3294322e666 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e34cbed578fd4fe2828f924cd9e73c8e'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:12.320086 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-9c8374cd-fc61-44af-b9ab-f3294322e666 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e34cbed578fd4fe2828f924cd9e73c8e'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:12.327099 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-9c8374cd-fc61-44af-b9ab-f3294322e666 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e34cbed578fd4fe2828f924cd9e73c8e'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:12.328379 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-9c8374cd-fc61-44af-b9ab-f3294322e666 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e34cbed578fd4fe2828f924cd9e73c8e'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:12.338011 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1274/2525] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:28:12 2022] DELETE /identity/v3/users/e34cbed578fd4fe2828f924cd9e73c8e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:12.611493 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360c211-44be-4c1f-994f-8d83ef0f412b admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:12.612976 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a360c211-44be-4c1f-994f-8d83ef0f412b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:12.627144 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360c211-44be-4c1f-994f-8d83ef0f412b admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:12.674701 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:12.694801 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e29861-349a-429c-a83e-053c6af7073a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:12.695683 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e29861-349a-429c-a83e-053c6af7073a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:12.695971 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e29861-349a-429c-a83e-053c6af7073a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:12.696248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e29861-349a-429c-a83e-053c6af7073a admin admin] PATH_INFO: `/v3/projects/ac77e64a12d9491b9233671cc5b52a28` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:12.697423 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e29861-349a-429c-a83e-053c6af7073a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac77e64a12d9491b9233671cc5b52a28)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:12.699653 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e29861-349a-429c-a83e-053c6af7073a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:12.743122 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d6e29861-349a-429c-a83e-053c6af7073a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac77e64a12d9491b9233671cc5b52a28', 'request_id': 'req-d6e29861-349a-429c-a83e-053c6af7073a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:12.748942 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d6e29861-349a-429c-a83e-053c6af7073a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac77e64a12d9491b9233671cc5b52a28', 'request_id': 'req-d6e29861-349a-429c-a83e-053c6af7073a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:12.757376 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d6e29861-349a-429c-a83e-053c6af7073a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac77e64a12d9491b9233671cc5b52a28', 'request_id': 'req-d6e29861-349a-429c-a83e-053c6af7073a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:12.767746 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d6e29861-349a-429c-a83e-053c6af7073a admin admin] The token cache is being invalidate because project ac77e64a12d9491b9233671cc5b52a28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:28:12.768029 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d6e29861-349a-429c-a83e-053c6af7073a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:12.768654 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d6e29861-349a-429c-a83e-053c6af7073a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:12.770188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1252/2526] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:28:12 2022] DELETE /identity/v3/projects/ac77e64a12d9491b9233671cc5b52a28 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:28:12.776357 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e2335-f51b-4163-9b18-ca45ee7455ef None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:12.776654 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e2335-f51b-4163-9b18-ca45ee7455ef None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:12.776930 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e2335-f51b-4163-9b18-ca45ee7455ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:12.806408 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-3c1e2335-f51b-4163-9b18-ca45ee7455ef None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:12.807645 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-3c1e2335-f51b-4163-9b18-ca45ee7455ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:12.857851 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:12.868599 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1275/2527] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:28:12.877257 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d7524-ba6d-46bf-9270-500bd4248258 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:12.877547 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d7524-ba6d-46bf-9270-500bd4248258 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:12.877821 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d7524-ba6d-46bf-9270-500bd4248258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:12.895696 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-279d7524-ba6d-46bf-9270-500bd4248258 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:12.896888 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-279d7524-ba6d-46bf-9270-500bd4248258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:12.905921 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1253/2528] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:12.910122 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e2335-f51b-4163-9b18-ca45ee7455ef None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:12.919732 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e2335-f51b-4163-9b18-ca45ee7455ef None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:12.924963 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f35d99-3171-41c7-b5df-a491a616e16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:12.925588 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f35d99-3171-41c7-b5df-a491a616e16b admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:12.925873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f35d99-3171-41c7-b5df-a491a616e16b admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:12.926151 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f35d99-3171-41c7-b5df-a491a616e16b admin admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:12.926872 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f35d99-3171-41c7-b5df-a491a616e16b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:28:12.927179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f35d99-3171-41c7-b5df-a491a616e16b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:12.928807 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f35d99-3171-41c7-b5df-a491a616e16b admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:12.940498 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1276/2529] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:28:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:28:12.944437 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d7524-ba6d-46bf-9270-500bd4248258 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:12.954084 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d7524-ba6d-46bf-9270-500bd4248258 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:12.958665 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5f9497-acbd-4872-a9b5-fec5f77dee1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:12.959317 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f9497-acbd-4872-a9b5-fec5f77dee1f admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:12.959607 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f9497-acbd-4872-a9b5-fec5f77dee1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:12.959882 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f9497-acbd-4872-a9b5-fec5f77dee1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:12.960585 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5f9497-acbd-4872-a9b5-fec5f77dee1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:12.962474 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5f9497-acbd-4872-a9b5-fec5f77dee1f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:12.987368 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1254/2530] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:28:12 2022] POST /identity/v3/projects => generated 387 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:28:12.991671 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f35d99-3171-41c7-b5df-a491a616e16b admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:13.002070 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f35d99-3171-41c7-b5df-a491a616e16b admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:13.061528 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b914bb-2037-4354-84d8-d786f0184e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:13.062698 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b914bb-2037-4354-84d8-d786f0184e3d admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:13.062981 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b914bb-2037-4354-84d8-d786f0184e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:13.063266 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b914bb-2037-4354-84d8-d786f0184e3d admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:13.064062 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b914bb-2037-4354-84d8-d786f0184e3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:13.065981 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b914bb-2037-4354-84d8-d786f0184e3d admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:13.087808 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1277/2531] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:28:12 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:28:13.092159 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5f9497-acbd-4872-a9b5-fec5f77dee1f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:13.103376 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5f9497-acbd-4872-a9b5-fec5f77dee1f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:13.108608 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb94590-a15a-483f-91b5-e47a49883fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:13.109288 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb94590-a15a-483f-91b5-e47a49883fa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:13.110359 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb94590-a15a-483f-91b5-e47a49883fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:13.110646 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb94590-a15a-483f-91b5-e47a49883fa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:13.111572 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb94590-a15a-483f-91b5-e47a49883fa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:13.113246 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb94590-a15a-483f-91b5-e47a49883fa2 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:13.124955 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1255/2532] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:28:13.128978 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b914bb-2037-4354-84d8-d786f0184e3d admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:13.139269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b914bb-2037-4354-84d8-d786f0184e3d admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:13.144621 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db94497-1fcc-4c0d-8722-4a90e949be33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:13.145277 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db94497-1fcc-4c0d-8722-4a90e949be33 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:13.145560 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db94497-1fcc-4c0d-8722-4a90e949be33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:13.145836 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db94497-1fcc-4c0d-8722-4a90e949be33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:13.146564 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db94497-1fcc-4c0d-8722-4a90e949be33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:13.148207 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db94497-1fcc-4c0d-8722-4a90e949be33 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:13.158565 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1278/2533] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:28:13.162693 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb94590-a15a-483f-91b5-e47a49883fa2 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:13.173094 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb94590-a15a-483f-91b5-e47a49883fa2 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:13.177682 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa40b091-4e94-4cc7-b09c-6fef38db7add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:13.178495 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa40b091-4e94-4cc7-b09c-6fef38db7add admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:13.178781 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa40b091-4e94-4cc7-b09c-6fef38db7add admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:13.179058 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa40b091-4e94-4cc7-b09c-6fef38db7add admin admin] PATH_INFO: `/v3/projects/9a17a4ea335f4728987e2aee6c5d65c4/users/468f43a8ee7a4b34935b9f56abe33c38/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:13.194119 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa40b091-4e94-4cc7-b09c-6fef38db7add admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a17a4ea335f4728987e2aee6c5d65c4, user_id=468f43a8ee7a4b34935b9f56abe33c38, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:13.196824 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa40b091-4e94-4cc7-b09c-6fef38db7add admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:13.209462 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1256/2534] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:28:13 2022] PUT /identity/v3/projects/9a17a4ea335f4728987e2aee6c5d65c4/users/468f43a8ee7a4b34935b9f56abe33c38/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:13.220983 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8fd8be-2710-4fd2-85fa-6d2035baa2a2 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:13.221446 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8fd8be-2710-4fd2-85fa-6d2035baa2a2 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:13.221576 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8fd8be-2710-4fd2-85fa-6d2035baa2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:13.247980 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-fd8fd8be-2710-4fd2-85fa-6d2035baa2a2 None None] MFA Rules not processed for user `468f43a8ee7a4b34935b9f56abe33c38`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:13.249780 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-fd8fd8be-2710-4fd2-85fa-6d2035baa2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:13.313354 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1279/2535] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:13.318107 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa40b091-4e94-4cc7-b09c-6fef38db7add admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:13.380465 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa40b091-4e94-4cc7-b09c-6fef38db7add admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:13.385402 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a9f52-819f-487f-b426-0f54c28a2edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:13.386058 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a9f52-819f-487f-b426-0f54c28a2edc admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:13.386348 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a9f52-819f-487f-b426-0f54c28a2edc admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:13.386634 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a9f52-819f-487f-b426-0f54c28a2edc admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:13.387408 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701a9f52-819f-487f-b426-0f54c28a2edc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:13.389329 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701a9f52-819f-487f-b426-0f54c28a2edc admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:13.400233 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:13.422046 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1257/2536] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:28:13 2022] POST /identity/v3/projects => generated 389 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:28:13.423145 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8fd8be-2710-4fd2-85fa-6d2035baa2a2 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:13.433765 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8fd8be-2710-4fd2-85fa-6d2035baa2a2 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:13.484113 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:13.492376 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c073e1c4-2850-4ad3-abc3-1ec72c6b7161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:13.493271 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073e1c4-2850-4ad3-abc3-1ec72c6b7161 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:13.493556 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073e1c4-2850-4ad3-abc3-1ec72c6b7161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:13.493832 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073e1c4-2850-4ad3-abc3-1ec72c6b7161 admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:13.494958 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c073e1c4-2850-4ad3-abc3-1ec72c6b7161 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:13.496677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c073e1c4-2850-4ad3-abc3-1ec72c6b7161 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:13.519033 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1280/2537] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:28:13 2022] POST /identity/v3/users => generated 301 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:28:13.523287 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a9f52-819f-487f-b426-0f54c28a2edc admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:13.534245 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a9f52-819f-487f-b426-0f54c28a2edc admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:13.539235 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ee96b-3551-4f8d-bd52-f1291661ced0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:13.539927 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ee96b-3551-4f8d-bd52-f1291661ced0 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:13.540218 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ee96b-3551-4f8d-bd52-f1291661ced0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:13.540501 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ee96b-3551-4f8d-bd52-f1291661ced0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:13.541228 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29ee96b-3551-4f8d-bd52-f1291661ced0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:13.542822 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29ee96b-3551-4f8d-bd52-f1291661ced0 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:13.553689 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1258/2538] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:28:13.558070 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c073e1c4-2850-4ad3-abc3-1ec72c6b7161 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:13.568942 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c073e1c4-2850-4ad3-abc3-1ec72c6b7161 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:13.574141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18062f5-fa06-419e-86d8-c7fe88d09bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:13.575027 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18062f5-fa06-419e-86d8-c7fe88d09bd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:13.575342 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18062f5-fa06-419e-86d8-c7fe88d09bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:13.575626 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18062f5-fa06-419e-86d8-c7fe88d09bd0 admin admin] PATH_INFO: `/v3/projects/6e8f462299794c04a363c882723064d0/users/472403f49f6f4d00842d44798c3308b4/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:13.593055 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18062f5-fa06-419e-86d8-c7fe88d09bd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e8f462299794c04a363c882723064d0, user_id=472403f49f6f4d00842d44798c3308b4, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:13.596109 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18062f5-fa06-419e-86d8-c7fe88d09bd0 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:13.610865 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1281/2539] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:28:13 2022] PUT /identity/v3/projects/6e8f462299794c04a363c882723064d0/users/472403f49f6f4d00842d44798c3308b4/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:13.615445 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ee96b-3551-4f8d-bd52-f1291661ced0 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:13.626064 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ee96b-3551-4f8d-bd52-f1291661ced0 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:13.681358 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff81c9-be22-418c-bd1c-a0768d723c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:13.682037 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff81c9-be22-418c-bd1c-a0768d723c79 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:13.682327 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff81c9-be22-418c-bd1c-a0768d723c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:13.682609 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff81c9-be22-418c-bd1c-a0768d723c79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:13.683364 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdff81c9-be22-418c-bd1c-a0768d723c79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:13.685061 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdff81c9-be22-418c-bd1c-a0768d723c79 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:13.696507 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1259/2540] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:13 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:28:13.700175 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18062f5-fa06-419e-86d8-c7fe88d09bd0 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:13.711488 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18062f5-fa06-419e-86d8-c7fe88d09bd0 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:13.716057 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a233d7f-979f-44f9-b9d6-1763da92c716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:13.716939 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a233d7f-979f-44f9-b9d6-1763da92c716 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:13.717212 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a233d7f-979f-44f9-b9d6-1763da92c716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:13.717483 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a233d7f-979f-44f9-b9d6-1763da92c716 admin admin] PATH_INFO: `/v3/projects/6e8f462299794c04a363c882723064d0/users/472403f49f6f4d00842d44798c3308b4/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:13.719635 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a233d7f-979f-44f9-b9d6-1763da92c716 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e8f462299794c04a363c882723064d0, user_id=472403f49f6f4d00842d44798c3308b4, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:13.722236 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a233d7f-979f-44f9-b9d6-1763da92c716 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:13.732779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1282/2541] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:28:13 2022] PUT /identity/v3/projects/6e8f462299794c04a363c882723064d0/users/472403f49f6f4d00842d44798c3308b4/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:13.743750 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09610b37-38a1-4334-81bd-40e6f28cc184 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:13.744044 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09610b37-38a1-4334-81bd-40e6f28cc184 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:13.744326 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09610b37-38a1-4334-81bd-40e6f28cc184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:13.777843 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-09610b37-38a1-4334-81bd-40e6f28cc184 None None] MFA Rules not processed for user `472403f49f6f4d00842d44798c3308b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:13.779152 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-09610b37-38a1-4334-81bd-40e6f28cc184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:13.844920 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1260/2542] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:13 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:13.855484 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a233d7f-979f-44f9-b9d6-1763da92c716 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:13.857147 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-2a233d7f-979f-44f9-b9d6-1763da92c716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:13.912295 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a233d7f-979f-44f9-b9d6-1763da92c716 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:13.916068 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c57bb5-8e26-47cc-a5b9-3b0858673337 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:13.916699 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c57bb5-8e26-47cc-a5b9-3b0858673337 service cinder] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:13.916982 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c57bb5-8e26-47cc-a5b9-3b0858673337 service cinder] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:13.917256 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c57bb5-8e26-47cc-a5b9-3b0858673337 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:13.926416 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c57bb5-8e26-47cc-a5b9-3b0858673337 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:13.927869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c57bb5-8e26-47cc-a5b9-3b0858673337 service cinder] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:13.937688 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1283/2543] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:28:13 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:28:14.062569 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09610b37-38a1-4334-81bd-40e6f28cc184 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:14.126947 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09610b37-38a1-4334-81bd-40e6f28cc184 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:14.131622 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f65318-3471-484c-af4f-e89c814d4a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:14.132572 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f65318-3471-484c-af4f-e89c814d4a4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:14.132754 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f65318-3471-484c-af4f-e89c814d4a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:14.133023 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f65318-3471-484c-af4f-e89c814d4a4d admin admin] PATH_INFO: `/v3/users/468f43a8ee7a4b34935b9f56abe33c38` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:14.134268 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f65318-3471-484c-af4f-e89c814d4a4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=468f43a8ee7a4b34935b9f56abe33c38)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:14.136212 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f65318-3471-484c-af4f-e89c814d4a4d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:14.138611 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:14.186027 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-11f65318-3471-484c-af4f-e89c814d4a4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '468f43a8ee7a4b34935b9f56abe33c38'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:14.194934 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-11f65318-3471-484c-af4f-e89c814d4a4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '468f43a8ee7a4b34935b9f56abe33c38'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:14.208273 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-11f65318-3471-484c-af4f-e89c814d4a4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '468f43a8ee7a4b34935b9f56abe33c38'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:14.208968 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-11f65318-3471-484c-af4f-e89c814d4a4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '468f43a8ee7a4b34935b9f56abe33c38'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:14.216557 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-11f65318-3471-484c-af4f-e89c814d4a4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '468f43a8ee7a4b34935b9f56abe33c38'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:14.222779 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1261/2544] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:28:14 2022] DELETE /identity/v3/users/468f43a8ee7a4b34935b9f56abe33c38 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:14.227320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215571a9-8165-452c-94df-71d15016d06a None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:14.227813 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215571a9-8165-452c-94df-71d15016d06a None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:14.227900 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215571a9-8165-452c-94df-71d15016d06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:14.251218 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-215571a9-8165-452c-94df-71d15016d06a None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:14.251218 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-215571a9-8165-452c-94df-71d15016d06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:14.310555 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1284/2545] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:28:14.320300 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f65318-3471-484c-af4f-e89c814d4a4d admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:14.321707 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-11f65318-3471-484c-af4f-e89c814d4a4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:14.378758 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f65318-3471-484c-af4f-e89c814d4a4d admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:14.382379 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee38247-2bba-4c4f-b704-75f5e8c0b641 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:14.383026 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee38247-2bba-4c4f-b704-75f5e8c0b641 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:14.383457 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee38247-2bba-4c4f-b704-75f5e8c0b641 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:14.383669 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee38247-2bba-4c4f-b704-75f5e8c0b641 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:14.394805 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee38247-2bba-4c4f-b704-75f5e8c0b641 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:14.396304 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee38247-2bba-4c4f-b704-75f5e8c0b641 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:14.406620 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1262/2546] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:28:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:28:14.670369 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215571a9-8165-452c-94df-71d15016d06a None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:14.672412 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-215571a9-8165-452c-94df-71d15016d06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:14.688358 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215571a9-8165-452c-94df-71d15016d06a None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:14.694083 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54515e11-046d-4bba-beb8-47f08943f42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:14.699277 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54515e11-046d-4bba-beb8-47f08943f42f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:14.699277 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54515e11-046d-4bba-beb8-47f08943f42f admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:14.699277 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54515e11-046d-4bba-beb8-47f08943f42f admin admin] PATH_INFO: `/v3/projects/9a17a4ea335f4728987e2aee6c5d65c4` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:14.699277 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54515e11-046d-4bba-beb8-47f08943f42f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a17a4ea335f4728987e2aee6c5d65c4)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:14.699277 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54515e11-046d-4bba-beb8-47f08943f42f admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:14.745582 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:14.749043 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-54515e11-046d-4bba-beb8-47f08943f42f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a17a4ea335f4728987e2aee6c5d65c4', 'request_id': 'req-54515e11-046d-4bba-beb8-47f08943f42f'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:14.756529 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-54515e11-046d-4bba-beb8-47f08943f42f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a17a4ea335f4728987e2aee6c5d65c4', 'request_id': 'req-54515e11-046d-4bba-beb8-47f08943f42f'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:14.763690 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-54515e11-046d-4bba-beb8-47f08943f42f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a17a4ea335f4728987e2aee6c5d65c4', 'request_id': 'req-54515e11-046d-4bba-beb8-47f08943f42f'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:14.773754 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-54515e11-046d-4bba-beb8-47f08943f42f admin admin] The token cache is being invalidate because project 9a17a4ea335f4728987e2aee6c5d65c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:28:14.774042 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-54515e11-046d-4bba-beb8-47f08943f42f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:14.774661 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-54515e11-046d-4bba-beb8-47f08943f42f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:14.776478 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1285/2547] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:28:14 2022] DELETE /identity/v3/projects/9a17a4ea335f4728987e2aee6c5d65c4 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:14.780432 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee38247-2bba-4c4f-b704-75f5e8c0b641 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:14.790927 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee38247-2bba-4c4f-b704-75f5e8c0b641 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:14.854182 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57602a2-6236-45da-a344-87ab18c0c4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:14.855052 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57602a2-6236-45da-a344-87ab18c0c4ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:14.855370 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57602a2-6236-45da-a344-87ab18c0c4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:14.855646 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57602a2-6236-45da-a344-87ab18c0c4ea admin admin] PATH_INFO: `/v3/users/472403f49f6f4d00842d44798c3308b4` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:14.856843 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57602a2-6236-45da-a344-87ab18c0c4ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=472403f49f6f4d00842d44798c3308b4)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:14.859424 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57602a2-6236-45da-a344-87ab18c0c4ea admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:14.892912 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:14.913203 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d57602a2-6236-45da-a344-87ab18c0c4ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '472403f49f6f4d00842d44798c3308b4'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:14.920876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d57602a2-6236-45da-a344-87ab18c0c4ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '472403f49f6f4d00842d44798c3308b4'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:14.933525 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d57602a2-6236-45da-a344-87ab18c0c4ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '472403f49f6f4d00842d44798c3308b4'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:14.934184 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d57602a2-6236-45da-a344-87ab18c0c4ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '472403f49f6f4d00842d44798c3308b4'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:14.941795 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d57602a2-6236-45da-a344-87ab18c0c4ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '472403f49f6f4d00842d44798c3308b4'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:14.948171 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1263/2548] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:28:14 2022] DELETE /identity/v3/users/472403f49f6f4d00842d44798c3308b4 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:15.210828 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54515e11-046d-4bba-beb8-47f08943f42f admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:15.212280 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-54515e11-046d-4bba-beb8-47f08943f42f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:15.226166 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54515e11-046d-4bba-beb8-47f08943f42f admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:15.282993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080163f4-2883-4545-b1e3-5e00bdf7002f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:15.283926 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080163f4-2883-4545-b1e3-5e00bdf7002f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:15.284208 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080163f4-2883-4545-b1e3-5e00bdf7002f admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:15.284481 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080163f4-2883-4545-b1e3-5e00bdf7002f admin admin] PATH_INFO: `/v3/projects/6e8f462299794c04a363c882723064d0` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:15.285643 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080163f4-2883-4545-b1e3-5e00bdf7002f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e8f462299794c04a363c882723064d0)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:15.287544 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080163f4-2883-4545-b1e3-5e00bdf7002f admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:15.331999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-080163f4-2883-4545-b1e3-5e00bdf7002f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e8f462299794c04a363c882723064d0', 'request_id': 'req-080163f4-2883-4545-b1e3-5e00bdf7002f'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:15.337863 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-080163f4-2883-4545-b1e3-5e00bdf7002f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e8f462299794c04a363c882723064d0', 'request_id': 'req-080163f4-2883-4545-b1e3-5e00bdf7002f'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:15.345583 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-080163f4-2883-4545-b1e3-5e00bdf7002f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e8f462299794c04a363c882723064d0', 'request_id': 'req-080163f4-2883-4545-b1e3-5e00bdf7002f'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:15.356090 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-080163f4-2883-4545-b1e3-5e00bdf7002f admin admin] The token cache is being invalidate because project 6e8f462299794c04a363c882723064d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:28:15.356381 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-080163f4-2883-4545-b1e3-5e00bdf7002f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:15.357011 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-080163f4-2883-4545-b1e3-5e00bdf7002f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:15.359120 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1286/2549] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:28:15 2022] DELETE /identity/v3/projects/6e8f462299794c04a363c882723064d0 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:15.364134 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ea0e4-77d1-4b4c-ae1e-e265396e8060 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:15.364587 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ea0e4-77d1-4b4c-ae1e-e265396e8060 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:15.364698 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ea0e4-77d1-4b4c-ae1e-e265396e8060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:15.381715 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-541ea0e4-77d1-4b4c-ae1e-e265396e8060 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:15.382908 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-541ea0e4-77d1-4b4c-ae1e-e265396e8060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:15.443562 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1264/2550] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:15.451232 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f82d9f-98c9-4234-9156-91e58b6d3f01 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:15.451575 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f82d9f-98c9-4234-9156-91e58b6d3f01 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:15.451860 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f82d9f-98c9-4234-9156-91e58b6d3f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:15.471008 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-c6f82d9f-98c9-4234-9156-91e58b6d3f01 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:15.472267 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-c6f82d9f-98c9-4234-9156-91e58b6d3f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:15.480477 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1287/2551] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:15.484908 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541ea0e4-77d1-4b4c-ae1e-e265396e8060 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:15.494893 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541ea0e4-77d1-4b4c-ae1e-e265396e8060 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:15.499671 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f63423-14d8-49f1-aa9c-949dcabbcae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:15.500423 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f63423-14d8-49f1-aa9c-949dcabbcae7 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:15.500780 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f63423-14d8-49f1-aa9c-949dcabbcae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:15.501132 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f63423-14d8-49f1-aa9c-949dcabbcae7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:15.501941 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f63423-14d8-49f1-aa9c-949dcabbcae7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:28:15.502325 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f63423-14d8-49f1-aa9c-949dcabbcae7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:15.504111 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f63423-14d8-49f1-aa9c-949dcabbcae7 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:15.520635 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1265/2552] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:28:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:28:15.521780 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f82d9f-98c9-4234-9156-91e58b6d3f01 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:15.531985 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f82d9f-98c9-4234-9156-91e58b6d3f01 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:15.537031 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448e8e7-3fc5-46a3-b5a4-7d219177a11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:15.537693 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448e8e7-3fc5-46a3-b5a4-7d219177a11a admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:15.537989 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448e8e7-3fc5-46a3-b5a4-7d219177a11a admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:15.538273 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448e8e7-3fc5-46a3-b5a4-7d219177a11a admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:15.538995 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5448e8e7-3fc5-46a3-b5a4-7d219177a11a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:15.540948 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5448e8e7-3fc5-46a3-b5a4-7d219177a11a admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:15.573040 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1288/2553] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:28:15 2022] POST /identity/v3/projects => generated 409 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:28:15.574188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f63423-14d8-49f1-aa9c-949dcabbcae7 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:15.584705 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f63423-14d8-49f1-aa9c-949dcabbcae7 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:15.640896 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e0ef7-93d8-4998-9ce4-c12f8a8ae3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:15.641555 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e0ef7-93d8-4998-9ce4-c12f8a8ae3ab admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:15.641843 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e0ef7-93d8-4998-9ce4-c12f8a8ae3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:15.642123 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e0ef7-93d8-4998-9ce4-c12f8a8ae3ab admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:15.642810 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430e0ef7-93d8-4998-9ce4-c12f8a8ae3ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:15.644773 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430e0ef7-93d8-4998-9ce4-c12f8a8ae3ab admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:15.665202 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1266/2554] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:28:15 2022] POST /identity/v3/users => generated 305 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:28:15.669497 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448e8e7-3fc5-46a3-b5a4-7d219177a11a admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:15.679880 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448e8e7-3fc5-46a3-b5a4-7d219177a11a admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:15.684453 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1765b54-179f-46a8-a261-fc59eaa96604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:15.685151 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1765b54-179f-46a8-a261-fc59eaa96604 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:15.685421 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1765b54-179f-46a8-a261-fc59eaa96604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:15.685699 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1765b54-179f-46a8-a261-fc59eaa96604 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:15.686420 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1765b54-179f-46a8-a261-fc59eaa96604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:15.688052 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1765b54-179f-46a8-a261-fc59eaa96604 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:15.699607 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1289/2555] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:28:15.704289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e0ef7-93d8-4998-9ce4-c12f8a8ae3ab admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:15.714299 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e0ef7-93d8-4998-9ce4-c12f8a8ae3ab admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:15.719567 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59efdd-4020-4688-9190-63152552455e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:15.720220 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59efdd-4020-4688-9190-63152552455e admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:15.720508 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59efdd-4020-4688-9190-63152552455e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:15.720786 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59efdd-4020-4688-9190-63152552455e admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:15.721540 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59efdd-4020-4688-9190-63152552455e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:15.723136 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59efdd-4020-4688-9190-63152552455e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:15.733028 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1267/2556] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:28:15.737246 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1765b54-179f-46a8-a261-fc59eaa96604 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:15.747518 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1765b54-179f-46a8-a261-fc59eaa96604 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:15.752628 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2dd7c-3b04-4c5f-9f8c-0ce522a923c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:15.753503 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2dd7c-3b04-4c5f-9f8c-0ce522a923c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:15.753787 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2dd7c-3b04-4c5f-9f8c-0ce522a923c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:15.754064 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2dd7c-3b04-4c5f-9f8c-0ce522a923c2 admin admin] PATH_INFO: `/v3/projects/672aba50f48e4725b3f1cf81aa2d33fa/users/042a4f04acd34cf69fba09e33ac85a63/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:15.769102 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a2dd7c-3b04-4c5f-9f8c-0ce522a923c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=672aba50f48e4725b3f1cf81aa2d33fa, user_id=042a4f04acd34cf69fba09e33ac85a63, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:15.771776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a2dd7c-3b04-4c5f-9f8c-0ce522a923c2 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:15.780671 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:15.786164 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1290/2557] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:28:15 2022] PUT /identity/v3/projects/672aba50f48e4725b3f1cf81aa2d33fa/users/042a4f04acd34cf69fba09e33ac85a63/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:15.796982 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ec5f9-e513-4181-af2b-15147ef35c1b None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:15.797277 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ec5f9-e513-4181-af2b-15147ef35c1b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:15.797560 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ec5f9-e513-4181-af2b-15147ef35c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:15.816265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:15.826390 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-202ec5f9-e513-4181-af2b-15147ef35c1b None None] MFA Rules not processed for user `042a4f04acd34cf69fba09e33ac85a63`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:15.829001 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-202ec5f9-e513-4181-af2b-15147ef35c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:15.891408 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1268/2558] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:15 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:15.895766 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2dd7c-3b04-4c5f-9f8c-0ce522a923c2 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:15.956053 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2dd7c-3b04-4c5f-9f8c-0ce522a923c2 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:15.960723 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96a629-4b97-49f8-b2d1-083f130ffc6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:15.961375 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96a629-4b97-49f8-b2d1-083f130ffc6e admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:15.961663 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96a629-4b97-49f8-b2d1-083f130ffc6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:15.961945 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96a629-4b97-49f8-b2d1-083f130ffc6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:15.962652 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb96a629-4b97-49f8-b2d1-083f130ffc6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:15.964664 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb96a629-4b97-49f8-b2d1-083f130ffc6e admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:15.989968 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1291/2559] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:28:15 2022] POST /identity/v3/projects => generated 407 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:28:15.993892 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ec5f9-e513-4181-af2b-15147ef35c1b None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:16.006878 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ec5f9-e513-4181-af2b-15147ef35c1b None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:16.075557 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f541efb2-615d-4dc5-a1c9-4147d9b2a66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:16.076344 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541efb2-615d-4dc5-a1c9-4147d9b2a66e admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:16.076730 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541efb2-615d-4dc5-a1c9-4147d9b2a66e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:16.077014 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541efb2-615d-4dc5-a1c9-4147d9b2a66e admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:16.078307 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f541efb2-615d-4dc5-a1c9-4147d9b2a66e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:16.080284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f541efb2-615d-4dc5-a1c9-4147d9b2a66e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:16.103533 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1269/2560] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:28:15 2022] POST /identity/v3/users => generated 310 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:28:16.105563 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96a629-4b97-49f8-b2d1-083f130ffc6e admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:16.117363 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96a629-4b97-49f8-b2d1-083f130ffc6e admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:16.122127 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4878f602-9f0a-4bcd-b115-a35755d9acfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:16.122798 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878f602-9f0a-4bcd-b115-a35755d9acfe admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:16.123086 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878f602-9f0a-4bcd-b115-a35755d9acfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:16.123403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878f602-9f0a-4bcd-b115-a35755d9acfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:16.124656 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4878f602-9f0a-4bcd-b115-a35755d9acfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:16.126269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4878f602-9f0a-4bcd-b115-a35755d9acfe admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:16.137107 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1292/2561] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:28:16.142275 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f541efb2-615d-4dc5-a1c9-4147d9b2a66e admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:16.152310 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f541efb2-615d-4dc5-a1c9-4147d9b2a66e admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:16.156883 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec35db65-bfdd-4ae9-8dd6-119d25164ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:16.157763 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35db65-bfdd-4ae9-8dd6-119d25164ed4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:16.158050 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35db65-bfdd-4ae9-8dd6-119d25164ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:16.158332 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35db65-bfdd-4ae9-8dd6-119d25164ed4 admin admin] PATH_INFO: `/v3/projects/0970c3823f85438e83139b3c8877511e/users/f139e6d39a144711aa430b4c169e7560/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:16.173115 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec35db65-bfdd-4ae9-8dd6-119d25164ed4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0970c3823f85438e83139b3c8877511e, user_id=f139e6d39a144711aa430b4c169e7560, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:16.175936 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec35db65-bfdd-4ae9-8dd6-119d25164ed4 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:16.189307 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1270/2562] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:28:16 2022] PUT /identity/v3/projects/0970c3823f85438e83139b3c8877511e/users/f139e6d39a144711aa430b4c169e7560/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:16.193294 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4878f602-9f0a-4bcd-b115-a35755d9acfe admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:16.203688 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4878f602-9f0a-4bcd-b115-a35755d9acfe admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:16.267532 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00cacde-8ad4-42a0-aa81-188e2e933f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:16.268208 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00cacde-8ad4-42a0-aa81-188e2e933f2b admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:16.268494 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00cacde-8ad4-42a0-aa81-188e2e933f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:16.268773 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00cacde-8ad4-42a0-aa81-188e2e933f2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:16.269497 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00cacde-8ad4-42a0-aa81-188e2e933f2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:16.271111 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00cacde-8ad4-42a0-aa81-188e2e933f2b admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:16.281141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1293/2563] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:28:16.285294 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec35db65-bfdd-4ae9-8dd6-119d25164ed4 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:16.295594 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec35db65-bfdd-4ae9-8dd6-119d25164ed4 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:16.300812 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c654d69-73cd-4682-bc86-84721df78672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:16.301694 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c654d69-73cd-4682-bc86-84721df78672 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:16.301982 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c654d69-73cd-4682-bc86-84721df78672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:16.302292 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c654d69-73cd-4682-bc86-84721df78672 admin admin] PATH_INFO: `/v3/projects/0970c3823f85438e83139b3c8877511e/users/f139e6d39a144711aa430b4c169e7560/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:16.304287 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c654d69-73cd-4682-bc86-84721df78672 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0970c3823f85438e83139b3c8877511e, user_id=f139e6d39a144711aa430b4c169e7560, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:16.307917 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c654d69-73cd-4682-bc86-84721df78672 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:16.320585 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1271/2564] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:28:16 2022] PUT /identity/v3/projects/0970c3823f85438e83139b3c8877511e/users/f139e6d39a144711aa430b4c169e7560/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:16.387760 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c647c1-4c10-4a98-81e7-7326f60af031 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:16.388348 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c647c1-4c10-4a98-81e7-7326f60af031 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:16.388348 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c647c1-4c10-4a98-81e7-7326f60af031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:16.415537 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-f0c647c1-4c10-4a98-81e7-7326f60af031 None None] MFA Rules not processed for user `f139e6d39a144711aa430b4c169e7560`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:16.416793 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-f0c647c1-4c10-4a98-81e7-7326f60af031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:16.479321 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:16.501483 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1294/2565] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:16 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:16.509168 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c654d69-73cd-4682-bc86-84721df78672 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:16.510960 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0c654d69-73cd-4682-bc86-84721df78672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:16.568193 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c654d69-73cd-4682-bc86-84721df78672 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:16.571844 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde088eb-517f-486d-98e6-380e6520fa9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:16.572484 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde088eb-517f-486d-98e6-380e6520fa9a service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:16.572768 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde088eb-517f-486d-98e6-380e6520fa9a service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:16.573107 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde088eb-517f-486d-98e6-380e6520fa9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:16.579146 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:16.584408 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde088eb-517f-486d-98e6-380e6520fa9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:16.586786 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde088eb-517f-486d-98e6-380e6520fa9a service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:16.598463 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1272/2566] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:28:16 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:28:16.783201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c647c1-4c10-4a98-81e7-7326f60af031 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:16.793273 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c647c1-4c10-4a98-81e7-7326f60af031 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:16.798639 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aafa649-6201-4e02-81cf-1a006851fac3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:16.799362 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aafa649-6201-4e02-81cf-1a006851fac3 service cinder] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:16.799800 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aafa649-6201-4e02-81cf-1a006851fac3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:16.800122 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aafa649-6201-4e02-81cf-1a006851fac3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:16.858237 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aafa649-6201-4e02-81cf-1a006851fac3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:16.860416 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aafa649-6201-4e02-81cf-1a006851fac3 service cinder] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:16.870224 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1295/2567] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:28:16 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:28:25.592537 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde088eb-517f-486d-98e6-380e6520fa9a service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:25.593927 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-dde088eb-517f-486d-98e6-380e6520fa9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:25.649085 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde088eb-517f-486d-98e6-380e6520fa9a service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:25.652839 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341cc89-4814-4b68-98b0-35ee2191fdc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:25.653470 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341cc89-4814-4b68-98b0-35ee2191fdc1 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:25.653755 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341cc89-4814-4b68-98b0-35ee2191fdc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:25.654030 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341cc89-4814-4b68-98b0-35ee2191fdc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:25.655814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4341cc89-4814-4b68-98b0-35ee2191fdc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:25.717244 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4341cc89-4814-4b68-98b0-35ee2191fdc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:25.718668 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4341cc89-4814-4b68-98b0-35ee2191fdc1 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:25.728122 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1273/2568] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:28:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:28:26.416825 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aafa649-6201-4e02-81cf-1a006851fac3 service cinder] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:26.428749 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aafa649-6201-4e02-81cf-1a006851fac3 service cinder] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:26.433629 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c30f6-a35f-414e-8c6e-ce91f10f8184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:26.434621 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c30f6-a35f-414e-8c6e-ce91f10f8184 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:26.434922 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c30f6-a35f-414e-8c6e-ce91f10f8184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:26.435215 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c30f6-a35f-414e-8c6e-ce91f10f8184 admin admin] PATH_INFO: `/v3/users/042a4f04acd34cf69fba09e33ac85a63` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:26.436548 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c30f6-a35f-414e-8c6e-ce91f10f8184 admin admin] RBAC: Authorizing `identity:delete_user(user_id=042a4f04acd34cf69fba09e33ac85a63)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:26.438507 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c30f6-a35f-414e-8c6e-ce91f10f8184 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:26.492480 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-d94c30f6-a35f-414e-8c6e-ce91f10f8184 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '042a4f04acd34cf69fba09e33ac85a63'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:26.500631 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-d94c30f6-a35f-414e-8c6e-ce91f10f8184 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '042a4f04acd34cf69fba09e33ac85a63'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:26.515715 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-d94c30f6-a35f-414e-8c6e-ce91f10f8184 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '042a4f04acd34cf69fba09e33ac85a63'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:26.523361 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-d94c30f6-a35f-414e-8c6e-ce91f10f8184 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '042a4f04acd34cf69fba09e33ac85a63'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:26.524158 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-d94c30f6-a35f-414e-8c6e-ce91f10f8184 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '042a4f04acd34cf69fba09e33ac85a63'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:26.530722 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1296/2569] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:28:26 2022] DELETE /identity/v3/users/042a4f04acd34cf69fba09e33ac85a63 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:26.535795 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786a4cf-c490-43bd-a803-fbd52b6548a8 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:26.536085 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786a4cf-c490-43bd-a803-fbd52b6548a8 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:26.536365 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786a4cf-c490-43bd-a803-fbd52b6548a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:26.554924 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-6786a4cf-c490-43bd-a803-fbd52b6548a8 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:26.556264 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6786a4cf-c490-43bd-a803-fbd52b6548a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:26.627201 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1274/2570] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:26.637256 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c30f6-a35f-414e-8c6e-ce91f10f8184 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:26.638737 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-d94c30f6-a35f-414e-8c6e-ce91f10f8184 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:26.703003 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c30f6-a35f-414e-8c6e-ce91f10f8184 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:26.707097 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b00aab-4aa3-45e7-a75e-e678d0839620 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:26.707780 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b00aab-4aa3-45e7-a75e-e678d0839620 service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:26.708063 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b00aab-4aa3-45e7-a75e-e678d0839620 service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:26.708394 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b00aab-4aa3-45e7-a75e-e678d0839620 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:26.718826 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b00aab-4aa3-45e7-a75e-e678d0839620 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:26.720403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b00aab-4aa3-45e7-a75e-e678d0839620 service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:26.731785 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1297/2571] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:28:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:28:27.012293 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6786a4cf-c490-43bd-a803-fbd52b6548a8 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:27.013950 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6786a4cf-c490-43bd-a803-fbd52b6548a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:27.029354 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6786a4cf-c490-43bd-a803-fbd52b6548a8 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:27.034370 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529ea2b-62b2-45c6-9c0c-63d509033ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:27.035242 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529ea2b-62b2-45c6-9c0c-63d509033ae7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:27.035569 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529ea2b-62b2-45c6-9c0c-63d509033ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:27.035849 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529ea2b-62b2-45c6-9c0c-63d509033ae7 admin admin] PATH_INFO: `/v3/projects/672aba50f48e4725b3f1cf81aa2d33fa` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:27.037038 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d529ea2b-62b2-45c6-9c0c-63d509033ae7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=672aba50f48e4725b3f1cf81aa2d33fa)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:27.039340 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d529ea2b-62b2-45c6-9c0c-63d509033ae7 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:27.088187 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:27.092377 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d529ea2b-62b2-45c6-9c0c-63d509033ae7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '672aba50f48e4725b3f1cf81aa2d33fa', 'request_id': 'req-d529ea2b-62b2-45c6-9c0c-63d509033ae7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:27.102853 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d529ea2b-62b2-45c6-9c0c-63d509033ae7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '672aba50f48e4725b3f1cf81aa2d33fa', 'request_id': 'req-d529ea2b-62b2-45c6-9c0c-63d509033ae7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:27.115414 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d529ea2b-62b2-45c6-9c0c-63d509033ae7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '672aba50f48e4725b3f1cf81aa2d33fa', 'request_id': 'req-d529ea2b-62b2-45c6-9c0c-63d509033ae7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:27.135047 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d529ea2b-62b2-45c6-9c0c-63d509033ae7 admin admin] The token cache is being invalidate because project 672aba50f48e4725b3f1cf81aa2d33fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:28:27.135805 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d529ea2b-62b2-45c6-9c0c-63d509033ae7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:27.137255 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d529ea2b-62b2-45c6-9c0c-63d509033ae7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:27.139961 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1275/2572] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:28:27 2022] DELETE /identity/v3/projects/672aba50f48e4725b3f1cf81aa2d33fa => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:27.144755 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b00aab-4aa3-45e7-a75e-e678d0839620 service neutron] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:27.155564 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b00aab-4aa3-45e7-a75e-e678d0839620 service neutron] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:27.161355 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:27.235050 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529ea2b-62b2-45c6-9c0c-63d509033ae7 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:27.236591 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d529ea2b-62b2-45c6-9c0c-63d509033ae7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:27.257436 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b5a51-0f48-4cda-b8c4-389448c112e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:27.258830 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b5a51-0f48-4cda-b8c4-389448c112e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:27.259285 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b5a51-0f48-4cda-b8c4-389448c112e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:27.259763 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b5a51-0f48-4cda-b8c4-389448c112e2 admin admin] PATH_INFO: `/v3/users/f139e6d39a144711aa430b4c169e7560` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:27.261226 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b5a51-0f48-4cda-b8c4-389448c112e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f139e6d39a144711aa430b4c169e7560)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:27.263349 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b5a51-0f48-4cda-b8c4-389448c112e2 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:27.332621 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529ea2b-62b2-45c6-9c0c-63d509033ae7 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:27.335477 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-159b5a51-0f48-4cda-b8c4-389448c112e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f139e6d39a144711aa430b4c169e7560'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:27.343437 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cbad52-9d57-4913-8b73-0a5052a412b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:27.344165 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cbad52-9d57-4913-8b73-0a5052a412b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:27.344481 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cbad52-9d57-4913-8b73-0a5052a412b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:27.344829 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cbad52-9d57-4913-8b73-0a5052a412b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:27.347179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-159b5a51-0f48-4cda-b8c4-389448c112e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f139e6d39a144711aa430b4c169e7560'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:27.353807 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c9cbad52-9d57-4913-8b73-0a5052a412b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:27.377114 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-159b5a51-0f48-4cda-b8c4-389448c112e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f139e6d39a144711aa430b4c169e7560'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:27.379103 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cbad52-9d57-4913-8b73-0a5052a412b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:27.380640 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cbad52-9d57-4913-8b73-0a5052a412b7 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:27.388759 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-159b5a51-0f48-4cda-b8c4-389448c112e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f139e6d39a144711aa430b4c169e7560'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:27.389553 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-159b5a51-0f48-4cda-b8c4-389448c112e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f139e6d39a144711aa430b4c169e7560'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:27.396951 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1298/2573] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:28:27 2022] DELETE /identity/v3/users/f139e6d39a144711aa430b4c169e7560 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:27.399536 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1276/2574] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:28:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 16:28:28.069335 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b5a51-0f48-4cda-b8c4-389448c112e2 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:28.070784 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-159b5a51-0f48-4cda-b8c4-389448c112e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:28.175586 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b5a51-0f48-4cda-b8c4-389448c112e2 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:28.180920 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb6f793-1359-4f31-9ba5-5e3c0c5fa7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:28.184008 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6f793-1359-4f31-9ba5-5e3c0c5fa7ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:28.184008 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6f793-1359-4f31-9ba5-5e3c0c5fa7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:28.184008 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6f793-1359-4f31-9ba5-5e3c0c5fa7ae admin admin] PATH_INFO: `/v3/projects/0970c3823f85438e83139b3c8877511e` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:28.186001 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb6f793-1359-4f31-9ba5-5e3c0c5fa7ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=0970c3823f85438e83139b3c8877511e)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:28.189339 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb6f793-1359-4f31-9ba5-5e3c0c5fa7ae admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:28.244157 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:28.267562 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-1eb6f793-1359-4f31-9ba5-5e3c0c5fa7ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0970c3823f85438e83139b3c8877511e', 'request_id': 'req-1eb6f793-1359-4f31-9ba5-5e3c0c5fa7ae'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:28.274580 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-1eb6f793-1359-4f31-9ba5-5e3c0c5fa7ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0970c3823f85438e83139b3c8877511e', 'request_id': 'req-1eb6f793-1359-4f31-9ba5-5e3c0c5fa7ae'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:28.284705 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-1eb6f793-1359-4f31-9ba5-5e3c0c5fa7ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0970c3823f85438e83139b3c8877511e', 'request_id': 'req-1eb6f793-1359-4f31-9ba5-5e3c0c5fa7ae'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:28.296137 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-1eb6f793-1359-4f31-9ba5-5e3c0c5fa7ae admin admin] The token cache is being invalidate because project 0970c3823f85438e83139b3c8877511e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:28:28.296552 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-1eb6f793-1359-4f31-9ba5-5e3c0c5fa7ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:28.297282 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-1eb6f793-1359-4f31-9ba5-5e3c0c5fa7ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:28.299720 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1299/2575] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:28:28 2022] DELETE /identity/v3/projects/0970c3823f85438e83139b3c8877511e => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:28.304276 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271650b2-65b8-4d77-b47a-6256b685037c None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:28.304748 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271650b2-65b8-4d77-b47a-6256b685037c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:28.304863 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271650b2-65b8-4d77-b47a-6256b685037c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:28.327060 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-271650b2-65b8-4d77-b47a-6256b685037c None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:28.329301 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-271650b2-65b8-4d77-b47a-6256b685037c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:28.408677 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1277/2576] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:28.416998 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52cf684-95f8-4ba1-9d85-17a7a434a42b None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:28.417293 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52cf684-95f8-4ba1-9d85-17a7a434a42b None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:28.417612 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52cf684-95f8-4ba1-9d85-17a7a434a42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:28.462833 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-d52cf684-95f8-4ba1-9d85-17a7a434a42b None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:28.467609 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-d52cf684-95f8-4ba1-9d85-17a7a434a42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:28.494045 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1300/2577] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:28.494246 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271650b2-65b8-4d77-b47a-6256b685037c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:28.512773 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271650b2-65b8-4d77-b47a-6256b685037c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:28.526479 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bfafe7-1655-426d-af58-7cb83e49afdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:28.529636 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bfafe7-1655-426d-af58-7cb83e49afdf admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:28.529945 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bfafe7-1655-426d-af58-7cb83e49afdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:28.530520 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bfafe7-1655-426d-af58-7cb83e49afdf admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:28.531008 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bfafe7-1655-426d-af58-7cb83e49afdf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:28:28.534460 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bfafe7-1655-426d-af58-7cb83e49afdf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:28.537611 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bfafe7-1655-426d-af58-7cb83e49afdf admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:28.573278 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1278/2578] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:28:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:28:28.577750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52cf684-95f8-4ba1-9d85-17a7a434a42b None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:28.593806 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52cf684-95f8-4ba1-9d85-17a7a434a42b None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:28.604680 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588eff3c-1873-4a34-a99e-0796b0c7c6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:28.605443 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588eff3c-1873-4a34-a99e-0796b0c7c6d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:28.607499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588eff3c-1873-4a34-a99e-0796b0c7c6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:28.607866 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588eff3c-1873-4a34-a99e-0796b0c7c6d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:28.608654 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588eff3c-1873-4a34-a99e-0796b0c7c6d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:28.611321 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588eff3c-1873-4a34-a99e-0796b0c7c6d0 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:28.664527 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1301/2579] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:28:28 2022] POST /identity/v3/projects => generated 381 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:28:28.667778 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bfafe7-1655-426d-af58-7cb83e49afdf admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:28.682100 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bfafe7-1655-426d-af58-7cb83e49afdf admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:28.779576 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:28.790823 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee61c59-5b45-4c70-a686-79c48d286d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:28.791942 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee61c59-5b45-4c70-a686-79c48d286d4d admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:28.792437 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee61c59-5b45-4c70-a686-79c48d286d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:28.792918 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee61c59-5b45-4c70-a686-79c48d286d4d admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:28.794227 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee61c59-5b45-4c70-a686-79c48d286d4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:28.796622 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee61c59-5b45-4c70-a686-79c48d286d4d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:28.837027 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1279/2580] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:28:28 2022] POST /identity/v3/users => generated 291 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:28:28.841647 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588eff3c-1873-4a34-a99e-0796b0c7c6d0 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:28.858220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588eff3c-1873-4a34-a99e-0796b0c7c6d0 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:28.868329 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289840c3-efe5-462c-9ebf-a7dd5490ef6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:28.869157 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289840c3-efe5-462c-9ebf-a7dd5490ef6e admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:28.869452 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289840c3-efe5-462c-9ebf-a7dd5490ef6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:28.869733 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289840c3-efe5-462c-9ebf-a7dd5490ef6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:28.870780 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289840c3-efe5-462c-9ebf-a7dd5490ef6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:28.872501 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289840c3-efe5-462c-9ebf-a7dd5490ef6e admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:28.892684 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1302/2581] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:28 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:28:28.897370 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee61c59-5b45-4c70-a686-79c48d286d4d admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:28.910462 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee61c59-5b45-4c70-a686-79c48d286d4d admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:28.915584 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10fd51-d4db-47f4-9214-8b00cae2a3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:28.916414 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10fd51-d4db-47f4-9214-8b00cae2a3eb admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:28.917434 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10fd51-d4db-47f4-9214-8b00cae2a3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:28.917729 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10fd51-d4db-47f4-9214-8b00cae2a3eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:28.918479 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10fd51-d4db-47f4-9214-8b00cae2a3eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:28.920138 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10fd51-d4db-47f4-9214-8b00cae2a3eb admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:28.934292 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1280/2582] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:28:28.939056 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289840c3-efe5-462c-9ebf-a7dd5490ef6e admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:28.959970 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289840c3-efe5-462c-9ebf-a7dd5490ef6e admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:28.976101 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f313aa9-117b-4736-b1b7-7881f9474700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:28.978291 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f313aa9-117b-4736-b1b7-7881f9474700 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:28.978654 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f313aa9-117b-4736-b1b7-7881f9474700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:28.983816 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f313aa9-117b-4736-b1b7-7881f9474700 admin admin] PATH_INFO: `/v3/projects/903fb8ac4c7947b6b98c30ee56ae9634/users/3b417045334f462aa4dc569f304629ca/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:29.012049 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f313aa9-117b-4736-b1b7-7881f9474700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=903fb8ac4c7947b6b98c30ee56ae9634, user_id=3b417045334f462aa4dc569f304629ca, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:29.018137 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f313aa9-117b-4736-b1b7-7881f9474700 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:29.038138 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1303/2583] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:28:28 2022] PUT /identity/v3/projects/903fb8ac4c7947b6b98c30ee56ae9634/users/3b417045334f462aa4dc569f304629ca/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:29.050213 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082c0cd-2565-4517-9dea-f254f3a95984 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:29.050858 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082c0cd-2565-4517-9dea-f254f3a95984 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:29.051215 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082c0cd-2565-4517-9dea-f254f3a95984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:29.097255 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f313aa9-117b-4736-b1b7-7881f9474700 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:29.098486 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-0f313aa9-117b-4736-b1b7-7881f9474700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:29.099881 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-7082c0cd-2565-4517-9dea-f254f3a95984 None None] MFA Rules not processed for user `3b417045334f462aa4dc569f304629ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:29.101104 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7082c0cd-2565-4517-9dea-f254f3a95984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:29.206015 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1281/2584] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:29.210534 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7082c0cd-2565-4517-9dea-f254f3a95984 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:29.218640 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f313aa9-117b-4736-b1b7-7881f9474700 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:29.227578 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7082c0cd-2565-4517-9dea-f254f3a95984 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:29.236556 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c1360-bf4f-45dc-bea9-44674d5c04cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:29.238232 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c1360-bf4f-45dc-bea9-44674d5c04cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:29.238622 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c1360-bf4f-45dc-bea9-44674d5c04cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:29.239066 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c1360-bf4f-45dc-bea9-44674d5c04cd admin admin] PATH_INFO: `/v3/users/94f91ae071b24e31a8ef8d50933ba4fb` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:29.240914 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1c1360-bf4f-45dc-bea9-44674d5c04cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=94f91ae071b24e31a8ef8d50933ba4fb)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:29.241781 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f970a-e4b7-4365-a89a-5ccf0423923b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:29.242553 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f970a-e4b7-4365-a89a-5ccf0423923b admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:29.243366 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1c1360-bf4f-45dc-bea9-44674d5c04cd admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:29.245072 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f970a-e4b7-4365-a89a-5ccf0423923b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:29.251400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f970a-e4b7-4365-a89a-5ccf0423923b admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:29.251400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28f970a-e4b7-4365-a89a-5ccf0423923b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:29.251400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28f970a-e4b7-4365-a89a-5ccf0423923b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:29.308354 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1282/2585] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:28:29 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:28:29.314301 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f970a-e4b7-4365-a89a-5ccf0423923b admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:29.338723 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-fb1c1360-bf4f-45dc-bea9-44674d5c04cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94f91ae071b24e31a8ef8d50933ba4fb'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:29.353602 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-fb1c1360-bf4f-45dc-bea9-44674d5c04cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94f91ae071b24e31a8ef8d50933ba4fb'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:29.358858 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=83,keystone:DELETE=13,keystone:INSERT=4 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:29.381412 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-fb1c1360-bf4f-45dc-bea9-44674d5c04cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94f91ae071b24e31a8ef8d50933ba4fb'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:29.390142 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-fb1c1360-bf4f-45dc-bea9-44674d5c04cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94f91ae071b24e31a8ef8d50933ba4fb'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:29.391576 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-fb1c1360-bf4f-45dc-bea9-44674d5c04cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94f91ae071b24e31a8ef8d50933ba4fb'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:29.399840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1304/2586] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:28:29 2022] DELETE /identity/v3/users/94f91ae071b24e31a8ef8d50933ba4fb => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:29.429698 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f970a-e4b7-4365-a89a-5ccf0423923b admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:29.503882 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:29.518436 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc0ffa9-ba6c-4593-b909-805053d89659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:29.519489 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0ffa9-ba6c-4593-b909-805053d89659 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:29.519932 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0ffa9-ba6c-4593-b909-805053d89659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:29.520426 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0ffa9-ba6c-4593-b909-805053d89659 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:29.521476 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc0ffa9-ba6c-4593-b909-805053d89659 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:29.523947 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc0ffa9-ba6c-4593-b909-805053d89659 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:29.565321 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1283/2587] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:28:29 2022] POST /identity/v3/users => generated 296 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:28:29.571165 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c1360-bf4f-45dc-bea9-44674d5c04cd admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:29.576316 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-fb1c1360-bf4f-45dc-bea9-44674d5c04cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:29.624995 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc0ffa9-ba6c-4593-b909-805053d89659 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:29.626850 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c1360-bf4f-45dc-bea9-44674d5c04cd admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:29.628121 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6cc0ffa9-ba6c-4593-b909-805053d89659 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:29.640276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e45ebd-6e50-4d61-9219-eed62287476f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:29.641438 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e45ebd-6e50-4d61-9219-eed62287476f admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:29.641879 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e45ebd-6e50-4d61-9219-eed62287476f admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:29.642310 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e45ebd-6e50-4d61-9219-eed62287476f admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:29.643206 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e45ebd-6e50-4d61-9219-eed62287476f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:29.645176 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e45ebd-6e50-4d61-9219-eed62287476f admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:29.651213 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc0ffa9-ba6c-4593-b909-805053d89659 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:29.657848 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e2250b-27b9-4d64-be48-dfd5b593547a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:29.659138 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2250b-27b9-4d64-be48-dfd5b593547a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:29.659533 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2250b-27b9-4d64-be48-dfd5b593547a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:29.659892 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2250b-27b9-4d64-be48-dfd5b593547a admin admin] PATH_INFO: `/v3/projects/ffa464e4d2264df68897a6d4b8dc7aab` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:29.662356 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e2250b-27b9-4d64-be48-dfd5b593547a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffa464e4d2264df68897a6d4b8dc7aab)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:29.664843 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e2250b-27b9-4d64-be48-dfd5b593547a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:29.683999 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1305/2588] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:28:29.687790 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e45ebd-6e50-4d61-9219-eed62287476f admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:29.713010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e45ebd-6e50-4d61-9219-eed62287476f admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:29.720000 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620197a-3a12-474e-8ed5-7a44fc00d8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:29.724538 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620197a-3a12-474e-8ed5-7a44fc00d8f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:29.725759 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620197a-3a12-474e-8ed5-7a44fc00d8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:29.726290 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620197a-3a12-474e-8ed5-7a44fc00d8f7 admin admin] PATH_INFO: `/v3/projects/c4e1e5ee67604b15890e820227c85483/users/8d5a390c15a24dadb54c55bcf0c6ca77/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:29.749757 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e620197a-3a12-474e-8ed5-7a44fc00d8f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4e1e5ee67604b15890e820227c85483, user_id=8d5a390c15a24dadb54c55bcf0c6ca77, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:29.753567 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e620197a-3a12-474e-8ed5-7a44fc00d8f7 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:29.779115 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1306/2589] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:28:29 2022] PUT /identity/v3/projects/c4e1e5ee67604b15890e820227c85483/users/8d5a390c15a24dadb54c55bcf0c6ca77/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:29.783433 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620197a-3a12-474e-8ed5-7a44fc00d8f7 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:29.801213 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-10e2250b-27b9-4d64-be48-dfd5b593547a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffa464e4d2264df68897a6d4b8dc7aab', 'request_id': 'req-10e2250b-27b9-4d64-be48-dfd5b593547a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:29.825447 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-10e2250b-27b9-4d64-be48-dfd5b593547a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffa464e4d2264df68897a6d4b8dc7aab', 'request_id': 'req-10e2250b-27b9-4d64-be48-dfd5b593547a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:29.836514 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-10e2250b-27b9-4d64-be48-dfd5b593547a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffa464e4d2264df68897a6d4b8dc7aab', 'request_id': 'req-10e2250b-27b9-4d64-be48-dfd5b593547a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:29.852971 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-10e2250b-27b9-4d64-be48-dfd5b593547a admin admin] The token cache is being invalidate because project ffa464e4d2264df68897a6d4b8dc7aab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:28:29.853405 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-10e2250b-27b9-4d64-be48-dfd5b593547a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:29.854187 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-10e2250b-27b9-4d64-be48-dfd5b593547a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:29.856264 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1284/2590] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:28:29 2022] DELETE /identity/v3/projects/ffa464e4d2264df68897a6d4b8dc7aab => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:29.862314 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd59671-5de1-49b3-885f-66423af94c73 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:29.863383 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd59671-5de1-49b3-885f-66423af94c73 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:29.863748 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd59671-5de1-49b3-885f-66423af94c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:29.883164 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-ecd59671-5de1-49b3-885f-66423af94c73 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:29.884431 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620197a-3a12-474e-8ed5-7a44fc00d8f7 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:29.884665 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ecd59671-5de1-49b3-885f-66423af94c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:29.949736 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab2ce9-ef74-4142-b799-03c06d921d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:29.950682 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1285/2591] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:28:29.950828 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab2ce9-ef74-4142-b799-03c06d921d8a admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:29.957106 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab2ce9-ef74-4142-b799-03c06d921d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:29.957406 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab2ce9-ef74-4142-b799-03c06d921d8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:29.958187 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab2ce9-ef74-4142-b799-03c06d921d8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:29.958498 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5613238-1bea-4cf2-b294-8d1f48359caf None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:29.958867 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5613238-1bea-4cf2-b294-8d1f48359caf None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:29.959394 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5613238-1bea-4cf2-b294-8d1f48359caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:29.959846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab2ce9-ef74-4142-b799-03c06d921d8a admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:29.974604 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1307/2592] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:28:29.976041 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab2ce9-ef74-4142-b799-03c06d921d8a admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:29.977860 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-41ab2ce9-ef74-4142-b799-03c06d921d8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:29.981997 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c5613238-1bea-4cf2-b294-8d1f48359caf None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:29.983360 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c5613238-1bea-4cf2-b294-8d1f48359caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:29.992017 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab2ce9-ef74-4142-b799-03c06d921d8a admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:29.992645 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1286/2593] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:28:29.996756 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5613238-1bea-4cf2-b294-8d1f48359caf None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:29.997121 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a829496-99e5-4157-a054-b1e7eb49a8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:29.998076 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a829496-99e5-4157-a054-b1e7eb49a8de admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:29.998431 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a829496-99e5-4157-a054-b1e7eb49a8de admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:29.998774 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a829496-99e5-4157-a054-b1e7eb49a8de admin admin] PATH_INFO: `/v3/projects/c4e1e5ee67604b15890e820227c85483/users/8d5a390c15a24dadb54c55bcf0c6ca77/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:30.001012 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a829496-99e5-4157-a054-b1e7eb49a8de admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4e1e5ee67604b15890e820227c85483, user_id=8d5a390c15a24dadb54c55bcf0c6ca77, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:30.004626 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a829496-99e5-4157-a054-b1e7eb49a8de admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:30.006215 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5613238-1bea-4cf2-b294-8d1f48359caf None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:30.011154 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f986bc7-b2df-4c8b-b0c8-b77455f0d35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:30.012120 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f986bc7-b2df-4c8b-b0c8-b77455f0d35c admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:30.012406 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f986bc7-b2df-4c8b-b0c8-b77455f0d35c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:30.012685 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f986bc7-b2df-4c8b-b0c8-b77455f0d35c admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:30.013420 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f986bc7-b2df-4c8b-b0c8-b77455f0d35c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:28:30.013819 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f986bc7-b2df-4c8b-b0c8-b77455f0d35c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:30.015643 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f986bc7-b2df-4c8b-b0c8-b77455f0d35c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:30.016873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1308/2594] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:28:29 2022] PUT /identity/v3/projects/c4e1e5ee67604b15890e820227c85483/users/8d5a390c15a24dadb54c55bcf0c6ca77/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:28:30.027021 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f294f0-6beb-40ae-9221-189f11be40d3 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:30.027256 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f294f0-6beb-40ae-9221-189f11be40d3 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:30.027571 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f294f0-6beb-40ae-9221-189f11be40d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:30.032844 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1287/2595] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:28:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:28:30.037327 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f986bc7-b2df-4c8b-b0c8-b77455f0d35c admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:30.048623 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f986bc7-b2df-4c8b-b0c8-b77455f0d35c admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:30.057322 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-90f294f0-6beb-40ae-9221-189f11be40d3 None None] MFA Rules not processed for user `8d5a390c15a24dadb54c55bcf0c6ca77`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:30.058551 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-90f294f0-6beb-40ae-9221-189f11be40d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:30.079880 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:30.094049 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:30.133093 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81108452-1520-47d3-a4c0-93db82e426a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:30.134041 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81108452-1520-47d3-a4c0-93db82e426a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:30.134426 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81108452-1520-47d3-a4c0-93db82e426a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:30.134809 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81108452-1520-47d3-a4c0-93db82e426a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:30.135790 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81108452-1520-47d3-a4c0-93db82e426a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:30.137877 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81108452-1520-47d3-a4c0-93db82e426a0 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:30.171107 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1309/2596] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:30 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:28:30.182530 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f294f0-6beb-40ae-9221-189f11be40d3 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:30.184243 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-90f294f0-6beb-40ae-9221-189f11be40d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:30.185686 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1288/2597] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:28:30 2022] POST /identity/v3/projects => generated 373 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:28:30.189500 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81108452-1520-47d3-a4c0-93db82e426a0 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:30.273954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f294f0-6beb-40ae-9221-189f11be40d3 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:30.280480 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81108452-1520-47d3-a4c0-93db82e426a0 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:30.285764 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fe51ed-5765-4e98-a819-381846c6978a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:30.286463 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fe51ed-5765-4e98-a819-381846c6978a admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:30.286753 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fe51ed-5765-4e98-a819-381846c6978a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:30.287037 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fe51ed-5765-4e98-a819-381846c6978a admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:30.287802 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fe51ed-5765-4e98-a819-381846c6978a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:30.290277 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fe51ed-5765-4e98-a819-381846c6978a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:30.316713 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1289/2598] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:28:30 2022] POST /identity/v3/users => generated 287 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:28:30.320680 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fe51ed-5765-4e98-a819-381846c6978a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:30.327559 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c91a231-66ab-4d38-a1d3-db6fce69c14c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:30.328218 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91a231-66ab-4d38-a1d3-db6fce69c14c service cinder] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:30.328505 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91a231-66ab-4d38-a1d3-db6fce69c14c service cinder] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:30.329320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91a231-66ab-4d38-a1d3-db6fce69c14c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:30.331942 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fe51ed-5765-4e98-a819-381846c6978a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:30.337305 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a73c672-9f58-4bfc-981e-5ad5d249aa91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:30.337988 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a73c672-9f58-4bfc-981e-5ad5d249aa91 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:30.338278 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a73c672-9f58-4bfc-981e-5ad5d249aa91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:30.338559 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a73c672-9f58-4bfc-981e-5ad5d249aa91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:30.339423 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a73c672-9f58-4bfc-981e-5ad5d249aa91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:30.341043 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a73c672-9f58-4bfc-981e-5ad5d249aa91 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:30.362003 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1290/2599] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:30 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:28:30.366050 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a73c672-9f58-4bfc-981e-5ad5d249aa91 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:30.376772 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a73c672-9f58-4bfc-981e-5ad5d249aa91 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:30.382162 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a909061e-1193-4ed7-9b61-c7935777d077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:30.382820 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909061e-1193-4ed7-9b61-c7935777d077 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:30.383113 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909061e-1193-4ed7-9b61-c7935777d077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:30.383422 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909061e-1193-4ed7-9b61-c7935777d077 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:30.384156 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a909061e-1193-4ed7-9b61-c7935777d077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:30.385755 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a909061e-1193-4ed7-9b61-c7935777d077 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:30.397692 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c91a231-66ab-4d38-a1d3-db6fce69c14c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:30.398868 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1291/2600] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:28:30.402426 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c91a231-66ab-4d38-a1d3-db6fce69c14c service cinder] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:30.403326 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a909061e-1193-4ed7-9b61-c7935777d077 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:30.414834 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1310/2601] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:28:30 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:28:30.415923 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a909061e-1193-4ed7-9b61-c7935777d077 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:30.421684 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374af705-5816-476f-9c1c-3c9c6c0c2228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:30.422662 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374af705-5816-476f-9c1c-3c9c6c0c2228 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:30.422951 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374af705-5816-476f-9c1c-3c9c6c0c2228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:30.423236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374af705-5816-476f-9c1c-3c9c6c0c2228 admin admin] PATH_INFO: `/v3/projects/6dd01e60a8c74079ae95475fdd080bbf/users/956771cacbd849a2b87bdc5048105bae/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:30.442481 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374af705-5816-476f-9c1c-3c9c6c0c2228 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dd01e60a8c74079ae95475fdd080bbf, user_id=956771cacbd849a2b87bdc5048105bae, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:30.445769 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374af705-5816-476f-9c1c-3c9c6c0c2228 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:30.461558 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1292/2602] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:28:30 2022] PUT /identity/v3/projects/6dd01e60a8c74079ae95475fdd080bbf/users/956771cacbd849a2b87bdc5048105bae/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:30.476193 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937c00b-8ec9-4f11-8f41-e11bbbe367c3 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:30.476710 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937c00b-8ec9-4f11-8f41-e11bbbe367c3 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:30.477067 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937c00b-8ec9-4f11-8f41-e11bbbe367c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:30.508968 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-9937c00b-8ec9-4f11-8f41-e11bbbe367c3 None None] MFA Rules not processed for user `956771cacbd849a2b87bdc5048105bae`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:30.510217 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-9937c00b-8ec9-4f11-8f41-e11bbbe367c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:30.520931 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:30.604006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1311/2603] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:30.617843 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374af705-5816-476f-9c1c-3c9c6c0c2228 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:30.619765 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-374af705-5816-476f-9c1c-3c9c6c0c2228 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:30.738207 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374af705-5816-476f-9c1c-3c9c6c0c2228 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:30.745102 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de2984-b68e-487a-a726-c5f71a660a96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:30.745936 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de2984-b68e-487a-a726-c5f71a660a96 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:30.746382 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de2984-b68e-487a-a726-c5f71a660a96 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:30.746847 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de2984-b68e-487a-a726-c5f71a660a96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:30.756630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68de2984-b68e-487a-a726-c5f71a660a96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:30.758256 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68de2984-b68e-487a-a726-c5f71a660a96 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:30.767626 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:30.774135 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1293/2604] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:28:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:28:30.902500 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937c00b-8ec9-4f11-8f41-e11bbbe367c3 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:31.037454 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de2984-b68e-487a-a726-c5f71a660a96 service nova] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:31.080994 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937c00b-8ec9-4f11-8f41-e11bbbe367c3 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:31.088010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c127296-b945-445e-94aa-c1b4638a919a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:31.088912 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c127296-b945-445e-94aa-c1b4638a919a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:31.089199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c127296-b945-445e-94aa-c1b4638a919a admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:31.089478 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c127296-b945-445e-94aa-c1b4638a919a admin admin] PATH_INFO: `/v3/users/3b417045334f462aa4dc569f304629ca` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:31.091007 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c127296-b945-445e-94aa-c1b4638a919a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b417045334f462aa4dc569f304629ca)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:31.093000 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c127296-b945-445e-94aa-c1b4638a919a admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:31.135179 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de2984-b68e-487a-a726-c5f71a660a96 service nova] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:31.148125 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bcace6-e0a6-4ae1-8dac-64e0bcf03fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:31.149556 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bcace6-e0a6-4ae1-8dac-64e0bcf03fce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:31.149893 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bcace6-e0a6-4ae1-8dac-64e0bcf03fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:31.150179 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bcace6-e0a6-4ae1-8dac-64e0bcf03fce admin admin] PATH_INFO: `/v3/users/956771cacbd849a2b87bdc5048105bae` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:31.152704 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bcace6-e0a6-4ae1-8dac-64e0bcf03fce admin admin] RBAC: Authorizing `identity:delete_user(user_id=956771cacbd849a2b87bdc5048105bae)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:31.155404 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bcace6-e0a6-4ae1-8dac-64e0bcf03fce admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:31.156786 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-7c127296-b945-445e-94aa-c1b4638a919a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b417045334f462aa4dc569f304629ca'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:31.171045 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-7c127296-b945-445e-94aa-c1b4638a919a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b417045334f462aa4dc569f304629ca'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:31.208534 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-7c127296-b945-445e-94aa-c1b4638a919a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b417045334f462aa4dc569f304629ca'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:31.221521 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-7c127296-b945-445e-94aa-c1b4638a919a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b417045334f462aa4dc569f304629ca'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:31.223353 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-7c127296-b945-445e-94aa-c1b4638a919a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b417045334f462aa4dc569f304629ca'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:31.231982 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1312/2605] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:28:30 2022] DELETE /identity/v3/users/3b417045334f462aa4dc569f304629ca => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:31.233011 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e0bcace6-e0a6-4ae1-8dac-64e0bcf03fce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '956771cacbd849a2b87bdc5048105bae'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:31.237541 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8d562-7b19-4987-a85b-0189edcdfbc3 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:31.237836 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8d562-7b19-4987-a85b-0189edcdfbc3 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:31.239120 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8d562-7b19-4987-a85b-0189edcdfbc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:31.254443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e0bcace6-e0a6-4ae1-8dac-64e0bcf03fce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '956771cacbd849a2b87bdc5048105bae'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:31.270053 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e0bcace6-e0a6-4ae1-8dac-64e0bcf03fce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '956771cacbd849a2b87bdc5048105bae'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:31.270688 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-3ef8d562-7b19-4987-a85b-0189edcdfbc3 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:31.270961 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e0bcace6-e0a6-4ae1-8dac-64e0bcf03fce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '956771cacbd849a2b87bdc5048105bae'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:31.272739 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-3ef8d562-7b19-4987-a85b-0189edcdfbc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:31.282237 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e0bcace6-e0a6-4ae1-8dac-64e0bcf03fce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '956771cacbd849a2b87bdc5048105bae'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:31.294752 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1294/2606] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:28:31 2022] DELETE /identity/v3/users/956771cacbd849a2b87bdc5048105bae => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:31.298978 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5716b-fb1d-4f71-ba5c-12ded1ff2b12 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:31.299702 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5716b-fb1d-4f71-ba5c-12ded1ff2b12 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:31.300065 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5716b-fb1d-4f71-ba5c-12ded1ff2b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:31.312571 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:31.333742 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-bef5716b-fb1d-4f71-ba5c-12ded1ff2b12 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:31.335963 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-bef5716b-fb1d-4f71-ba5c-12ded1ff2b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:31.365542 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1313/2607] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:31.376102 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8d562-7b19-4987-a85b-0189edcdfbc3 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:31.377916 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-3ef8d562-7b19-4987-a85b-0189edcdfbc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:31.418134 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1295/2608] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:28:31.429099 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef5716b-fb1d-4f71-ba5c-12ded1ff2b12 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:31.430567 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-bef5716b-fb1d-4f71-ba5c-12ded1ff2b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:31.455874 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:31.461646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8d562-7b19-4987-a85b-0189edcdfbc3 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:31.523791 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef5716b-fb1d-4f71-ba5c-12ded1ff2b12 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:31.529630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c029c9-e620-41db-ba48-fd83c8b19016 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:31.530622 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c029c9-e620-41db-ba48-fd83c8b19016 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:31.531135 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c029c9-e620-41db-ba48-fd83c8b19016 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:31.532217 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c029c9-e620-41db-ba48-fd83c8b19016 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:31.548165 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c029c9-e620-41db-ba48-fd83c8b19016 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:31.549682 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c029c9-e620-41db-ba48-fd83c8b19016 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:31.565243 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1296/2609] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:28:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:28:31.586084 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60179be-81de-4409-9fa1-f6ce1df9b5db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:31.586995 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60179be-81de-4409-9fa1-f6ce1df9b5db service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:31.587632 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60179be-81de-4409-9fa1-f6ce1df9b5db service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:31.588092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60179be-81de-4409-9fa1-f6ce1df9b5db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:31.606172 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60179be-81de-4409-9fa1-f6ce1df9b5db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:31.608262 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60179be-81de-4409-9fa1-f6ce1df9b5db service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:31.619367 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c029c9-e620-41db-ba48-fd83c8b19016 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:31.620498 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d4c029c9-e620-41db-ba48-fd83c8b19016 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:31.626592 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1314/2610] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:28:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 16:28:31.644966 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c029c9-e620-41db-ba48-fd83c8b19016 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:31.650642 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c93250-e418-486a-985d-ffc8b0258c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:31.652001 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c93250-e418-486a-985d-ffc8b0258c93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:31.652514 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c93250-e418-486a-985d-ffc8b0258c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:31.652738 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c93250-e418-486a-985d-ffc8b0258c93 admin admin] PATH_INFO: `/v3/users/ef7c1e449b864b50b3c71c39f19968a0` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:31.654773 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c93250-e418-486a-985d-ffc8b0258c93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef7c1e449b864b50b3c71c39f19968a0)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:31.657672 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c93250-e418-486a-985d-ffc8b0258c93 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:31.714264 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-55c93250-e418-486a-985d-ffc8b0258c93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef7c1e449b864b50b3c71c39f19968a0'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:31.722040 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-55c93250-e418-486a-985d-ffc8b0258c93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef7c1e449b864b50b3c71c39f19968a0'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:31.734667 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-55c93250-e418-486a-985d-ffc8b0258c93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef7c1e449b864b50b3c71c39f19968a0'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:31.735441 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-55c93250-e418-486a-985d-ffc8b0258c93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef7c1e449b864b50b3c71c39f19968a0'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:31.742542 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-55c93250-e418-486a-985d-ffc8b0258c93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef7c1e449b864b50b3c71c39f19968a0'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:31.748877 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1297/2611] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:28:31 2022] DELETE /identity/v3/users/ef7c1e449b864b50b3c71c39f19968a0 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:32.068903 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60179be-81de-4409-9fa1-f6ce1df9b5db service neutron] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:32.070650 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-e60179be-81de-4409-9fa1-f6ce1df9b5db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:32.084913 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60179be-81de-4409-9fa1-f6ce1df9b5db service neutron] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:32.096119 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c93250-e418-486a-985d-ffc8b0258c93 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:32.097728 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-55c93250-e418-486a-985d-ffc8b0258c93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:32.175183 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f13eb-8e5e-4989-b538-cb3a3e2bc19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:32.176700 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f13eb-8e5e-4989-b538-cb3a3e2bc19e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:32.177063 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f13eb-8e5e-4989-b538-cb3a3e2bc19e admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:32.177501 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f13eb-8e5e-4989-b538-cb3a3e2bc19e admin admin] PATH_INFO: `/v3/projects/6dd01e60a8c74079ae95475fdd080bbf` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:32.179133 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091f13eb-8e5e-4989-b538-cb3a3e2bc19e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dd01e60a8c74079ae95475fdd080bbf)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:32.181939 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091f13eb-8e5e-4989-b538-cb3a3e2bc19e admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:32.202930 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c93250-e418-486a-985d-ffc8b0258c93 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:32.208512 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e7c54-7368-4eab-8b79-4e3ff65754bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:32.209393 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e7c54-7368-4eab-8b79-4e3ff65754bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:32.209676 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e7c54-7368-4eab-8b79-4e3ff65754bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:32.209950 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e7c54-7368-4eab-8b79-4e3ff65754bc admin admin] PATH_INFO: `/v3/projects/c6758d3ab63e4ace8d2d32a4b391d942` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:32.211324 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85e7c54-7368-4eab-8b79-4e3ff65754bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6758d3ab63e4ace8d2d32a4b391d942)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:32.213605 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85e7c54-7368-4eab-8b79-4e3ff65754bc admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:32.221014 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:32.242197 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-091f13eb-8e5e-4989-b538-cb3a3e2bc19e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dd01e60a8c74079ae95475fdd080bbf', 'request_id': 'req-091f13eb-8e5e-4989-b538-cb3a3e2bc19e'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:32.250031 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-091f13eb-8e5e-4989-b538-cb3a3e2bc19e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dd01e60a8c74079ae95475fdd080bbf', 'request_id': 'req-091f13eb-8e5e-4989-b538-cb3a3e2bc19e'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:32.255683 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:32.260730 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-091f13eb-8e5e-4989-b538-cb3a3e2bc19e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dd01e60a8c74079ae95475fdd080bbf', 'request_id': 'req-091f13eb-8e5e-4989-b538-cb3a3e2bc19e'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:32.267254 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f85e7c54-7368-4eab-8b79-4e3ff65754bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6758d3ab63e4ace8d2d32a4b391d942', 'request_id': 'req-f85e7c54-7368-4eab-8b79-4e3ff65754bc'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:32.272604 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-091f13eb-8e5e-4989-b538-cb3a3e2bc19e admin admin] The token cache is being invalidate because project 6dd01e60a8c74079ae95475fdd080bbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:28:32.272896 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-091f13eb-8e5e-4989-b538-cb3a3e2bc19e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:32.273650 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-091f13eb-8e5e-4989-b538-cb3a3e2bc19e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:32.273791 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f85e7c54-7368-4eab-8b79-4e3ff65754bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6758d3ab63e4ace8d2d32a4b391d942', 'request_id': 'req-f85e7c54-7368-4eab-8b79-4e3ff65754bc'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:32.275345 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1315/2612] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:28:32 2022] DELETE /identity/v3/projects/6dd01e60a8c74079ae95475fdd080bbf => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:32.276506 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f13eb-8e5e-4989-b538-cb3a3e2bc19e admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:32.278042 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-091f13eb-8e5e-4989-b538-cb3a3e2bc19e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:32.282680 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f85e7c54-7368-4eab-8b79-4e3ff65754bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6758d3ab63e4ace8d2d32a4b391d942', 'request_id': 'req-f85e7c54-7368-4eab-8b79-4e3ff65754bc'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:32.294488 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f85e7c54-7368-4eab-8b79-4e3ff65754bc admin admin] The token cache is being invalidate because project c6758d3ab63e4ace8d2d32a4b391d942 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:28:32.294931 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f85e7c54-7368-4eab-8b79-4e3ff65754bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:32.295677 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f85e7c54-7368-4eab-8b79-4e3ff65754bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:32.297367 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1298/2613] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:28:32 2022] DELETE /identity/v3/projects/c6758d3ab63e4ace8d2d32a4b391d942 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:28:32.301401 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934121f-8896-4fc0-ac60-994c23ee1290 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:32.301401 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934121f-8896-4fc0-ac60-994c23ee1290 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:32.301401 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934121f-8896-4fc0-ac60-994c23ee1290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:32.319084 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-9934121f-8896-4fc0-ac60-994c23ee1290 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:32.320075 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9934121f-8896-4fc0-ac60-994c23ee1290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:32.345606 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f13eb-8e5e-4989-b538-cb3a3e2bc19e admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:32.378787 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1299/2614] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:32.380088 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e8789-721e-47e1-98f9-b857d2053474 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:32.380377 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e8789-721e-47e1-98f9-b857d2053474 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:32.380654 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e8789-721e-47e1-98f9-b857d2053474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:32.399234 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-fb4e8789-721e-47e1-98f9-b857d2053474 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:32.400478 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-fb4e8789-721e-47e1-98f9-b857d2053474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:32.408608 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1300/2615] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:32.408733 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0405b293-1c78-4931-8ea7-2c6c5558e67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:32.409511 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0405b293-1c78-4931-8ea7-2c6c5558e67a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:32.409799 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde9532-8de3-40e8-8801-b935b8cc64d3 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:32.409918 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0405b293-1c78-4931-8ea7-2c6c5558e67a admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:32.410091 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0405b293-1c78-4931-8ea7-2c6c5558e67a admin admin] PATH_INFO: `/v3/projects/903fb8ac4c7947b6b98c30ee56ae9634` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:32.410209 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde9532-8de3-40e8-8801-b935b8cc64d3 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:32.410380 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde9532-8de3-40e8-8801-b935b8cc64d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:32.411270 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0405b293-1c78-4931-8ea7-2c6c5558e67a admin admin] RBAC: Authorizing `identity:delete_project(project_id=903fb8ac4c7947b6b98c30ee56ae9634)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:32.413240 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0405b293-1c78-4931-8ea7-2c6c5558e67a admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:32.427231 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-7bde9532-8de3-40e8-8801-b935b8cc64d3 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:32.428457 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7bde9532-8de3-40e8-8801-b935b8cc64d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:32.436457 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1301/2616] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:32.437660 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c8c4f-7c28-4e50-979a-413137d8cc97 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:32.437949 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c8c4f-7c28-4e50-979a-413137d8cc97 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:32.438744 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c8c4f-7c28-4e50-979a-413137d8cc97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:32.458075 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-df5c8c4f-7c28-4e50-979a-413137d8cc97 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:32.459267 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-df5c8c4f-7c28-4e50-979a-413137d8cc97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:32.464844 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0405b293-1c78-4931-8ea7-2c6c5558e67a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '903fb8ac4c7947b6b98c30ee56ae9634', 'request_id': 'req-0405b293-1c78-4931-8ea7-2c6c5558e67a'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:32.471467 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0405b293-1c78-4931-8ea7-2c6c5558e67a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '903fb8ac4c7947b6b98c30ee56ae9634', 'request_id': 'req-0405b293-1c78-4931-8ea7-2c6c5558e67a'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:32.479226 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0405b293-1c78-4931-8ea7-2c6c5558e67a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '903fb8ac4c7947b6b98c30ee56ae9634', 'request_id': 'req-0405b293-1c78-4931-8ea7-2c6c5558e67a'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:32.484086 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:32.492316 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0405b293-1c78-4931-8ea7-2c6c5558e67a admin admin] The token cache is being invalidate because project 903fb8ac4c7947b6b98c30ee56ae9634 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:28:32.493052 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0405b293-1c78-4931-8ea7-2c6c5558e67a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:32.493744 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0405b293-1c78-4931-8ea7-2c6c5558e67a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:32.495802 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1316/2617] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:28:32 2022] DELETE /identity/v3/projects/903fb8ac4c7947b6b98c30ee56ae9634 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:32.496622 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0405b293-1c78-4931-8ea7-2c6c5558e67a admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:32.498035 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-0405b293-1c78-4931-8ea7-2c6c5558e67a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:32.524013 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1302/2618] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:32.527782 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5c8c4f-7c28-4e50-979a-413137d8cc97 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:32.527782 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-df5c8c4f-7c28-4e50-979a-413137d8cc97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:32.538326 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5c8c4f-7c28-4e50-979a-413137d8cc97 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:32.543174 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5422e154-f16d-4007-96da-fb5b51a945aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:32.544190 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422e154-f16d-4007-96da-fb5b51a945aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:32.544493 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422e154-f16d-4007-96da-fb5b51a945aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:32.544750 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422e154-f16d-4007-96da-fb5b51a945aa admin admin] PATH_INFO: `/v3/users/8d5a390c15a24dadb54c55bcf0c6ca77` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:32.545935 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5422e154-f16d-4007-96da-fb5b51a945aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d5a390c15a24dadb54c55bcf0c6ca77)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:32.548730 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5422e154-f16d-4007-96da-fb5b51a945aa admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:32.564815 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0405b293-1c78-4931-8ea7-2c6c5558e67a admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:32.569831 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0850a7e-be7b-47b9-b2d2-a17a7cf5ec3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:32.570450 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0850a7e-be7b-47b9-b2d2-a17a7cf5ec3d admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:32.570763 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0850a7e-be7b-47b9-b2d2-a17a7cf5ec3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:32.571043 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0850a7e-be7b-47b9-b2d2-a17a7cf5ec3d admin admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:32.571833 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0850a7e-be7b-47b9-b2d2-a17a7cf5ec3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:28:32.572145 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0850a7e-be7b-47b9-b2d2-a17a7cf5ec3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:32.574134 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0850a7e-be7b-47b9-b2d2-a17a7cf5ec3d admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:32.588218 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1317/2619] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:28:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:28:32.592328 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0850a7e-be7b-47b9-b2d2-a17a7cf5ec3d admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:32.596379 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:32.598081 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5422e154-f16d-4007-96da-fb5b51a945aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d5a390c15a24dadb54c55bcf0c6ca77'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:32.603041 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0850a7e-be7b-47b9-b2d2-a17a7cf5ec3d admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:32.606573 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5422e154-f16d-4007-96da-fb5b51a945aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d5a390c15a24dadb54c55bcf0c6ca77'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:32.610020 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eddbb8-efb7-4787-8408-d0ed54ef3ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:32.611519 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eddbb8-efb7-4787-8408-d0ed54ef3ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:32.611953 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eddbb8-efb7-4787-8408-d0ed54ef3ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:32.612371 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eddbb8-efb7-4787-8408-d0ed54ef3ce3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:32.613483 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eddbb8-efb7-4787-8408-d0ed54ef3ce3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:28:32.613960 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eddbb8-efb7-4787-8408-d0ed54ef3ce3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:32.616522 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eddbb8-efb7-4787-8408-d0ed54ef3ce3 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:32.620310 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5422e154-f16d-4007-96da-fb5b51a945aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d5a390c15a24dadb54c55bcf0c6ca77'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:32.620979 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5422e154-f16d-4007-96da-fb5b51a945aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d5a390c15a24dadb54c55bcf0c6ca77'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:32.628742 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5422e154-f16d-4007-96da-fb5b51a945aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d5a390c15a24dadb54c55bcf0c6ca77'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:32.630739 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1318/2620] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:28:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:28:32.631685 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eddbb8-efb7-4787-8408-d0ed54ef3ce3 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:32.633459 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-94eddbb8-efb7-4787-8408-d0ed54ef3ce3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:32.635470 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1303/2621] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:28:32 2022] DELETE /identity/v3/users/8d5a390c15a24dadb54c55bcf0c6ca77 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:32.636379 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5422e154-f16d-4007-96da-fb5b51a945aa admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:32.637979 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5422e154-f16d-4007-96da-fb5b51a945aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:32.647412 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eddbb8-efb7-4787-8408-d0ed54ef3ce3 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:32.718863 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5422e154-f16d-4007-96da-fb5b51a945aa admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:32.723659 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a8dbb7-e1aa-46fc-8a09-59cff809058c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:32.724568 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d29210-4405-45d3-81c2-29bdba13271a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:32.724846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8dbb7-e1aa-46fc-8a09-59cff809058c admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:32.725156 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8dbb7-e1aa-46fc-8a09-59cff809058c admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:32.725272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d29210-4405-45d3-81c2-29bdba13271a admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:32.725441 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8dbb7-e1aa-46fc-8a09-59cff809058c admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:32.725555 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d29210-4405-45d3-81c2-29bdba13271a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:32.725801 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d29210-4405-45d3-81c2-29bdba13271a admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:32.726172 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a8dbb7-e1aa-46fc-8a09-59cff809058c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:32.726501 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d29210-4405-45d3-81c2-29bdba13271a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:32.728077 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a8dbb7-e1aa-46fc-8a09-59cff809058c admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:32.728447 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d29210-4405-45d3-81c2-29bdba13271a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:32.758134 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1304/2622] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:28:32 2022] POST /identity/v3/projects => generated 371 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:28:32.758345 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1319/2623] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:28:32 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:28:32.762475 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d29210-4405-45d3-81c2-29bdba13271a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:32.762637 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a8dbb7-e1aa-46fc-8a09-59cff809058c admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:32.838818 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a8dbb7-e1aa-46fc-8a09-59cff809058c admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:32.845004 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d29210-4405-45d3-81c2-29bdba13271a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:32.845480 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3511ae4f-c8bb-4180-a03c-5777dc37900b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:32.846237 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3511ae4f-c8bb-4180-a03c-5777dc37900b admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:32.846628 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3511ae4f-c8bb-4180-a03c-5777dc37900b admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:32.846990 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3511ae4f-c8bb-4180-a03c-5777dc37900b admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:32.847771 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3511ae4f-c8bb-4180-a03c-5777dc37900b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:32.849750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3511ae4f-c8bb-4180-a03c-5777dc37900b admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:32.850832 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee8634-c2e7-4e20-8574-08c5bfa71bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:32.851695 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee8634-c2e7-4e20-8574-08c5bfa71bb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:32.852131 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee8634-c2e7-4e20-8574-08c5bfa71bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:32.852558 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee8634-c2e7-4e20-8574-08c5bfa71bb2 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:32.853420 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee8634-c2e7-4e20-8574-08c5bfa71bb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:32.855494 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee8634-c2e7-4e20-8574-08c5bfa71bb2 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:32.858651 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:32.878541 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1305/2624] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:28:32 2022] POST /identity/v3/users => generated 286 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:28:32.883164 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1320/2625] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:28:32 2022] POST /identity/v3/users => generated 287 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:28:32.883369 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee8634-c2e7-4e20-8574-08c5bfa71bb2 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:32.884523 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3511ae4f-c8bb-4180-a03c-5777dc37900b admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:32.895057 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3511ae4f-c8bb-4180-a03c-5777dc37900b admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:32.895271 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee8634-c2e7-4e20-8574-08c5bfa71bb2 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:32.900440 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae83e3-d12e-4d74-bc9d-29f0e6003a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:32.900607 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f96a5-0b64-47e8-8149-c61bb6ebf5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:32.901363 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f96a5-0b64-47e8-8149-c61bb6ebf5db admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:32.901529 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae83e3-d12e-4d74-bc9d-29f0e6003a2b admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:32.901630 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f96a5-0b64-47e8-8149-c61bb6ebf5db admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:32.901843 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae83e3-d12e-4d74-bc9d-29f0e6003a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:32.902048 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f96a5-0b64-47e8-8149-c61bb6ebf5db admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:32.902164 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae83e3-d12e-4d74-bc9d-29f0e6003a2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:32.902780 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15f96a5-0b64-47e8-8149-c61bb6ebf5db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:32.902908 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae83e3-d12e-4d74-bc9d-29f0e6003a2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:32.904428 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15f96a5-0b64-47e8-8149-c61bb6ebf5db admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:32.904729 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae83e3-d12e-4d74-bc9d-29f0e6003a2b admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:32.921715 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1306/2626] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:28:32.925015 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1321/2627] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:28:32.928428 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f96a5-0b64-47e8-8149-c61bb6ebf5db admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:32.928895 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae83e3-d12e-4d74-bc9d-29f0e6003a2b admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:32.940277 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f96a5-0b64-47e8-8149-c61bb6ebf5db admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:32.941553 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae83e3-d12e-4d74-bc9d-29f0e6003a2b admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:32.946121 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93538c5-3514-4559-afa7-754ce599124f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:32.946320 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b1e145-5697-461f-9ae7-f15be32d1bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:32.946976 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93538c5-3514-4559-afa7-754ce599124f admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:32.947113 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1e145-5697-461f-9ae7-f15be32d1bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:32.947259 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1e145-5697-461f-9ae7-f15be32d1bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:32.947460 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93538c5-3514-4559-afa7-754ce599124f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:32.947576 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1e145-5697-461f-9ae7-f15be32d1bd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:32.947854 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93538c5-3514-4559-afa7-754ce599124f admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:32.948291 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b1e145-5697-461f-9ae7-f15be32d1bd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:32.948769 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93538c5-3514-4559-afa7-754ce599124f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:32.949886 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b1e145-5697-461f-9ae7-f15be32d1bd0 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:32.950599 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93538c5-3514-4559-afa7-754ce599124f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:32.963167 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1307/2628] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:28:32.964053 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93538c5-3514-4559-afa7-754ce599124f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:32.966488 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c93538c5-3514-4559-afa7-754ce599124f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:32.966677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1322/2629] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:28:32.967721 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b1e145-5697-461f-9ae7-f15be32d1bd0 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:32.979595 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b1e145-5697-461f-9ae7-f15be32d1bd0 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:32.982658 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93538c5-3514-4559-afa7-754ce599124f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:32.984508 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dade80-6444-45be-8c52-e15df2c46c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:32.985418 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dade80-6444-45be-8c52-e15df2c46c58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:32.985703 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dade80-6444-45be-8c52-e15df2c46c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:32.985983 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dade80-6444-45be-8c52-e15df2c46c58 admin admin] PATH_INFO: `/v3/projects/6b68fed95706473cb2c06c72558da3e2/users/06dc59a28a4a4709909054470058064a/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:32.987443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf2c1d6-3ff4-40d3-9cee-5dc8cb1edccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:32.988276 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2c1d6-3ff4-40d3-9cee-5dc8cb1edccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:32.988558 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2c1d6-3ff4-40d3-9cee-5dc8cb1edccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:32.988833 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2c1d6-3ff4-40d3-9cee-5dc8cb1edccc admin admin] PATH_INFO: `/v3/projects/c4e1e5ee67604b15890e820227c85483` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:32.990016 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf2c1d6-3ff4-40d3-9cee-5dc8cb1edccc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4e1e5ee67604b15890e820227c85483)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:32.992002 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf2c1d6-3ff4-40d3-9cee-5dc8cb1edccc admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:33.002658 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dade80-6444-45be-8c52-e15df2c46c58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b68fed95706473cb2c06c72558da3e2, user_id=06dc59a28a4a4709909054470058064a, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:33.003775 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:33.005355 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dade80-6444-45be-8c52-e15df2c46c58 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:33.019439 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1323/2630] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:28:32 2022] PUT /identity/v3/projects/6b68fed95706473cb2c06c72558da3e2/users/06dc59a28a4a4709909054470058064a/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:33.020378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dade80-6444-45be-8c52-e15df2c46c58 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:33.048950 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-8bf2c1d6-3ff4-40d3-9cee-5dc8cb1edccc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4e1e5ee67604b15890e820227c85483', 'request_id': 'req-8bf2c1d6-3ff4-40d3-9cee-5dc8cb1edccc'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:33.055382 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-8bf2c1d6-3ff4-40d3-9cee-5dc8cb1edccc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4e1e5ee67604b15890e820227c85483', 'request_id': 'req-8bf2c1d6-3ff4-40d3-9cee-5dc8cb1edccc'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:33.062908 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-8bf2c1d6-3ff4-40d3-9cee-5dc8cb1edccc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4e1e5ee67604b15890e820227c85483', 'request_id': 'req-8bf2c1d6-3ff4-40d3-9cee-5dc8cb1edccc'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:33.074379 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-8bf2c1d6-3ff4-40d3-9cee-5dc8cb1edccc admin admin] The token cache is being invalidate because project c4e1e5ee67604b15890e820227c85483 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:28:33.074697 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-8bf2c1d6-3ff4-40d3-9cee-5dc8cb1edccc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:33.075406 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-8bf2c1d6-3ff4-40d3-9cee-5dc8cb1edccc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:33.076978 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1308/2631] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:28:32 2022] DELETE /identity/v3/projects/c4e1e5ee67604b15890e820227c85483 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:33.078270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24111e4f-0b97-44a5-bb79-504f51d0fc1c None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:33.078563 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24111e4f-0b97-44a5-bb79-504f51d0fc1c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:33.079493 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24111e4f-0b97-44a5-bb79-504f51d0fc1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:33.086151 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dade80-6444-45be-8c52-e15df2c46c58 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:33.105393 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-24111e4f-0b97-44a5-bb79-504f51d0fc1c None None] MFA Rules not processed for user `06dc59a28a4a4709909054470058064a`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:33.106569 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-24111e4f-0b97-44a5-bb79-504f51d0fc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:33.142968 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa83a1c-64dd-4abf-8ece-62f8e9a8ad22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:33.143894 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa83a1c-64dd-4abf-8ece-62f8e9a8ad22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:33.144184 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa83a1c-64dd-4abf-8ece-62f8e9a8ad22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:33.144461 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa83a1c-64dd-4abf-8ece-62f8e9a8ad22 admin admin] PATH_INFO: `/v3/projects/c492f0fdd7d54d0da0b68ffd73132596/users/b81617c1e30240649db32ec6e2b11227/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:33.160367 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa83a1c-64dd-4abf-8ece-62f8e9a8ad22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c492f0fdd7d54d0da0b68ffd73132596, user_id=b81617c1e30240649db32ec6e2b11227, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:33.163003 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa83a1c-64dd-4abf-8ece-62f8e9a8ad22 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:33.175669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1324/2632] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:28:33 2022] PUT /identity/v3/projects/c492f0fdd7d54d0da0b68ffd73132596/users/b81617c1e30240649db32ec6e2b11227/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:33.175950 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1309/2633] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:33.176976 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc72916-78fd-4651-9d11-81d20ca77cc9 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:33.177272 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc72916-78fd-4651-9d11-81d20ca77cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:33.177549 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc72916-78fd-4651-9d11-81d20ca77cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:33.192137 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd30966-3fa5-43df-91bf-ccf445c1060f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:33.192435 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd30966-3fa5-43df-91bf-ccf445c1060f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:33.192709 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd30966-3fa5-43df-91bf-ccf445c1060f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:33.196018 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-1cc72916-78fd-4651-9d11-81d20ca77cc9 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:33.197283 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-1cc72916-78fd-4651-9d11-81d20ca77cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:33.220389 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-4fd30966-3fa5-43df-91bf-ccf445c1060f None None] MFA Rules not processed for user `b81617c1e30240649db32ec6e2b11227`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:33.221628 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4fd30966-3fa5-43df-91bf-ccf445c1060f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:33.228697 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:33.264876 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1325/2634] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:33.265724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc72916-78fd-4651-9d11-81d20ca77cc9 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:33.267672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-1cc72916-78fd-4651-9d11-81d20ca77cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:33.294793 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1310/2635] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:28:33.296114 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5224d9-d193-4f6e-b264-7ca8f9dada36 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:33.296410 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5224d9-d193-4f6e-b264-7ca8f9dada36 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:33.296692 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5224d9-d193-4f6e-b264-7ca8f9dada36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:33.316202 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-ac5224d9-d193-4f6e-b264-7ca8f9dada36 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:33.317519 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ac5224d9-d193-4f6e-b264-7ca8f9dada36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:33.325617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1311/2636] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:33.326387 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5224d9-d193-4f6e-b264-7ca8f9dada36 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:33.327672 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ac5224d9-d193-4f6e-b264-7ca8f9dada36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:33.339686 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5224d9-d193-4f6e-b264-7ca8f9dada36 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:33.344564 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff9bc3c-035d-4d29-a159-ad8c537a3c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:33.345239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9bc3c-035d-4d29-a159-ad8c537a3c97 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:33.345529 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9bc3c-035d-4d29-a159-ad8c537a3c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:33.345808 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9bc3c-035d-4d29-a159-ad8c537a3c97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:33.346488 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff9bc3c-035d-4d29-a159-ad8c537a3c97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:33.348424 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff9bc3c-035d-4d29-a159-ad8c537a3c97 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:33.356710 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:33.372206 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc72916-78fd-4651-9d11-81d20ca77cc9 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:33.379199 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1312/2637] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:28:33 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:28:33.379199 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff9bc3c-035d-4d29-a159-ad8c537a3c97 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:33.380000 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b120a6-6f07-4d17-8735-2d851c9fa58a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:33.380000 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b120a6-6f07-4d17-8735-2d851c9fa58a service nova] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:33.380000 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b120a6-6f07-4d17-8735-2d851c9fa58a service nova] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:33.380287 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b120a6-6f07-4d17-8735-2d851c9fa58a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:33.437295 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b120a6-6f07-4d17-8735-2d851c9fa58a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:33.437552 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff9bc3c-035d-4d29-a159-ad8c537a3c97 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:33.439461 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b120a6-6f07-4d17-8735-2d851c9fa58a service nova] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:33.442675 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69dbc59-889a-4484-a9ce-7a3497afa103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:33.443345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69dbc59-889a-4484-a9ce-7a3497afa103 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:33.443658 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69dbc59-889a-4484-a9ce-7a3497afa103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:33.443978 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69dbc59-889a-4484-a9ce-7a3497afa103 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:33.444751 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69dbc59-889a-4484-a9ce-7a3497afa103 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:28:33.445065 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69dbc59-889a-4484-a9ce-7a3497afa103 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:33.446759 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69dbc59-889a-4484-a9ce-7a3497afa103 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:33.448924 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1326/2638] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:28:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:28:33.453063 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b120a6-6f07-4d17-8735-2d851c9fa58a service nova] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:33.459269 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1313/2639] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:28:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:28:33.463185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69dbc59-889a-4484-a9ce-7a3497afa103 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:33.471045 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b120a6-6f07-4d17-8735-2d851c9fa58a service nova] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:33.472870 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69dbc59-889a-4484-a9ce-7a3497afa103 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:33.475747 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a449f78-cff2-45be-b0ef-06705d63f099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:33.476403 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a449f78-cff2-45be-b0ef-06705d63f099 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:33.476715 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a449f78-cff2-45be-b0ef-06705d63f099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:33.476990 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a449f78-cff2-45be-b0ef-06705d63f099 admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:33.477706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a449f78-cff2-45be-b0ef-06705d63f099 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:33.478416 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea55616-d103-417f-b909-be8282c6094b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:33.479149 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea55616-d103-417f-b909-be8282c6094b admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:33.479534 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea55616-d103-417f-b909-be8282c6094b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:33.479645 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a449f78-cff2-45be-b0ef-06705d63f099 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:33.479820 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea55616-d103-417f-b909-be8282c6094b admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:33.480524 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea55616-d103-417f-b909-be8282c6094b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:33.482462 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea55616-d103-417f-b909-be8282c6094b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:33.505568 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1327/2640] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:28:33 2022] POST /identity/v3/users => generated 293 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:28:33.506705 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a449f78-cff2-45be-b0ef-06705d63f099 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:33.515235 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1314/2641] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:28:33 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:28:33.518222 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a449f78-cff2-45be-b0ef-06705d63f099 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:33.519385 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea55616-d103-417f-b909-be8282c6094b admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:33.546756 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:33.582881 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a54a3b-f443-42ba-950a-08278da27a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:33.583816 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a54a3b-f443-42ba-950a-08278da27a48 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:33.584245 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a54a3b-f443-42ba-950a-08278da27a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:33.584532 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a54a3b-f443-42ba-950a-08278da27a48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:33.585264 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a54a3b-f443-42ba-950a-08278da27a48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:33.587016 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a54a3b-f443-42ba-950a-08278da27a48 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:33.591247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea55616-d103-417f-b909-be8282c6094b admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:33.596199 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ec33d-f735-4daf-b7c6-ae599ce37528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:33.596952 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ec33d-f735-4daf-b7c6-ae599ce37528 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:33.597291 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ec33d-f735-4daf-b7c6-ae599ce37528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:33.597676 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ec33d-f735-4daf-b7c6-ae599ce37528 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:33.598552 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947ec33d-f735-4daf-b7c6-ae599ce37528 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:33.598696 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1328/2642] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:28:33.600592 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947ec33d-f735-4daf-b7c6-ae599ce37528 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:33.603102 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a54a3b-f443-42ba-950a-08278da27a48 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:33.614552 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a54a3b-f443-42ba-950a-08278da27a48 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:33.620434 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e92ba-c107-48f0-b425-324867e7db05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:33.621338 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e92ba-c107-48f0-b425-324867e7db05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:33.621639 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e92ba-c107-48f0-b425-324867e7db05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:33.621926 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e92ba-c107-48f0-b425-324867e7db05 admin admin] PATH_INFO: `/v3/projects/585ef734e97d48a69e15c06b30772988/users/a5a402682108401cade304441a87393a/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:33.630659 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1315/2643] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:28:33 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:28:33.632804 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ec33d-f735-4daf-b7c6-ae599ce37528 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:33.638756 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e92ba-c107-48f0-b425-324867e7db05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=585ef734e97d48a69e15c06b30772988, user_id=a5a402682108401cade304441a87393a, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:33.641493 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e92ba-c107-48f0-b425-324867e7db05 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:33.645701 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ec33d-f735-4daf-b7c6-ae599ce37528 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:33.651541 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b719ad2-a20d-4025-b558-79a6f66bf7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:33.652252 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b719ad2-a20d-4025-b558-79a6f66bf7cc admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:33.652591 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b719ad2-a20d-4025-b558-79a6f66bf7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:33.652879 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b719ad2-a20d-4025-b558-79a6f66bf7cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:33.653720 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b719ad2-a20d-4025-b558-79a6f66bf7cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:33.655584 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b719ad2-a20d-4025-b558-79a6f66bf7cc admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:33.659408 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1329/2644] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:28:33 2022] PUT /identity/v3/projects/585ef734e97d48a69e15c06b30772988/users/a5a402682108401cade304441a87393a/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:28:33.663162 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e92ba-c107-48f0-b425-324867e7db05 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:33.667335 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1316/2645] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:28:33.670794 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b719ad2-a20d-4025-b558-79a6f66bf7cc admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:33.683829 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b719ad2-a20d-4025-b558-79a6f66bf7cc admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:33.732190 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:33.751419 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e92ba-c107-48f0-b425-324867e7db05 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:33.756103 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f80e14-2ae5-4615-919d-0376d43679a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:33.756864 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f80e14-2ae5-4615-919d-0376d43679a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:33.757214 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f80e14-2ae5-4615-919d-0376d43679a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:33.757557 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f80e14-2ae5-4615-919d-0376d43679a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:33.758353 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f80e14-2ae5-4615-919d-0376d43679a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:33.760058 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f80e14-2ae5-4615-919d-0376d43679a0 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:33.771728 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1330/2646] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:28:33.773351 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec42fc8-433f-4d6c-ad14-aa4a4cbe4056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:33.774152 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec42fc8-433f-4d6c-ad14-aa4a4cbe4056 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:33.774853 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec42fc8-433f-4d6c-ad14-aa4a4cbe4056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:33.775049 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec42fc8-433f-4d6c-ad14-aa4a4cbe4056 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:33.775724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f80e14-2ae5-4615-919d-0376d43679a0 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:33.776185 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec42fc8-433f-4d6c-ad14-aa4a4cbe4056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:33.778730 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec42fc8-433f-4d6c-ad14-aa4a4cbe4056 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:33.786616 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f80e14-2ae5-4615-919d-0376d43679a0 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:33.795467 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1317/2647] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:28:33.797671 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59ba8b-a275-4894-bcbc-a11852008dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:33.798701 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59ba8b-a275-4894-bcbc-a11852008dc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:33.799615 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59ba8b-a275-4894-bcbc-a11852008dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:33.799851 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec42fc8-433f-4d6c-ad14-aa4a4cbe4056 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:33.799964 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59ba8b-a275-4894-bcbc-a11852008dc5 admin admin] PATH_INFO: `/v3/projects/585ef734e97d48a69e15c06b30772988/users/a5a402682108401cade304441a87393a/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:33.802384 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59ba8b-a275-4894-bcbc-a11852008dc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=585ef734e97d48a69e15c06b30772988, user_id=a5a402682108401cade304441a87393a, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:33.805393 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59ba8b-a275-4894-bcbc-a11852008dc5 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:33.810487 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec42fc8-433f-4d6c-ad14-aa4a4cbe4056 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:33.838609 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1331/2648] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:28:33 2022] PUT /identity/v3/projects/585ef734e97d48a69e15c06b30772988/users/a5a402682108401cade304441a87393a/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:28:33.841772 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ae358-0e40-4322-a6a7-bd4593da17ea None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:33.842176 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ae358-0e40-4322-a6a7-bd4593da17ea None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:33.843224 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ae358-0e40-4322-a6a7-bd4593da17ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:33.878549 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-ad5ae358-0e40-4322-a6a7-bd4593da17ea None None] MFA Rules not processed for user `a5a402682108401cade304441a87393a`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:33.879860 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-ad5ae358-0e40-4322-a6a7-bd4593da17ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:33.889924 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f172056c-d80c-4f09-8d44-f0d855cacc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:33.890809 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172056c-d80c-4f09-8d44-f0d855cacc43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:33.891116 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172056c-d80c-4f09-8d44-f0d855cacc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:33.892037 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172056c-d80c-4f09-8d44-f0d855cacc43 admin admin] PATH_INFO: `/v3/projects/caa595780261498997d276a7269332cd/users/28de27ba3a664986bbb18cf2a1e4bced/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:33.907093 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f172056c-d80c-4f09-8d44-f0d855cacc43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=caa595780261498997d276a7269332cd, user_id=28de27ba3a664986bbb18cf2a1e4bced, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:33.910502 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f172056c-d80c-4f09-8d44-f0d855cacc43 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:33.923708 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1318/2649] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:28:33 2022] PUT /identity/v3/projects/caa595780261498997d276a7269332cd/users/28de27ba3a664986bbb18cf2a1e4bced/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:33.934764 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c932f0-bc6d-4e8c-8e96-e92028071aad None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:33.935044 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c932f0-bc6d-4e8c-8e96-e92028071aad None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:33.935359 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c932f0-bc6d-4e8c-8e96-e92028071aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:33.950973 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1332/2650] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:33 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:33.962513 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ae358-0e40-4322-a6a7-bd4593da17ea None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:33.963074 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b5c932f0-bc6d-4e8c-8e96-e92028071aad None None] MFA Rules not processed for user `28de27ba3a664986bbb18cf2a1e4bced`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:33.964252 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-ad5ae358-0e40-4322-a6a7-bd4593da17ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:33.964379 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b5c932f0-bc6d-4e8c-8e96-e92028071aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:33.975394 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:34.029288 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ae358-0e40-4322-a6a7-bd4593da17ea None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:34.038066 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1319/2651] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:28:34.049137 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c932f0-bc6d-4e8c-8e96-e92028071aad None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:34.050538 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b5c932f0-bc6d-4e8c-8e96-e92028071aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:34.077373 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5879b2f4-daa7-4780-8f6a-57e6ba4e4fe1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:34.078026 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5879b2f4-daa7-4780-8f6a-57e6ba4e4fe1 service cinder] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:34.078314 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5879b2f4-daa7-4780-8f6a-57e6ba4e4fe1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:34.078619 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5879b2f4-daa7-4780-8f6a-57e6ba4e4fe1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:34.104806 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:34.108979 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c932f0-bc6d-4e8c-8e96-e92028071aad None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:34.114088 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dcae81-8c95-408f-a6a7-5dead70851c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:34.114737 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dcae81-8c95-408f-a6a7-5dead70851c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:34.115036 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dcae81-8c95-408f-a6a7-5dead70851c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:34.115441 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dcae81-8c95-408f-a6a7-5dead70851c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:34.126126 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dcae81-8c95-408f-a6a7-5dead70851c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:34.127859 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dcae81-8c95-408f-a6a7-5dead70851c2 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:34.136941 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1320/2652] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:28:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 16:28:34.144419 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5879b2f4-daa7-4780-8f6a-57e6ba4e4fe1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:34.145858 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5879b2f4-daa7-4780-8f6a-57e6ba4e4fe1 service cinder] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:34.156767 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1333/2653] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:28:33 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 16:28:34.227914 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dcae81-8c95-408f-a6a7-5dead70851c2 service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:34.237410 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dcae81-8c95-408f-a6a7-5dead70851c2 service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:34.241515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99b6bf-03ad-4273-879a-b85665d3bd59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:34.242219 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99b6bf-03ad-4273-879a-b85665d3bd59 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:34.242568 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99b6bf-03ad-4273-879a-b85665d3bd59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:34.242844 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99b6bf-03ad-4273-879a-b85665d3bd59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:34.306342 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e99b6bf-03ad-4273-879a-b85665d3bd59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:34.307845 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e99b6bf-03ad-4273-879a-b85665d3bd59 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:34.317876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1321/2654] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:28:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:28:36.167804 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5879b2f4-daa7-4780-8f6a-57e6ba4e4fe1 service cinder] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:36.167804 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-5879b2f4-daa7-4780-8f6a-57e6ba4e4fe1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:36.232770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5879b2f4-daa7-4780-8f6a-57e6ba4e4fe1 service cinder] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:36.237380 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cffdc5-d5b5-48f9-a3e8-fa1fbf1995b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:36.238243 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cffdc5-d5b5-48f9-a3e8-fa1fbf1995b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:36.238525 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cffdc5-d5b5-48f9-a3e8-fa1fbf1995b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:36.238799 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cffdc5-d5b5-48f9-a3e8-fa1fbf1995b1 admin admin] PATH_INFO: `/v3/users/06dc59a28a4a4709909054470058064a` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:36.239983 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cffdc5-d5b5-48f9-a3e8-fa1fbf1995b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06dc59a28a4a4709909054470058064a)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:36.241924 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cffdc5-d5b5-48f9-a3e8-fa1fbf1995b1 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:36.257380 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:36.287043 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-72cffdc5-d5b5-48f9-a3e8-fa1fbf1995b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06dc59a28a4a4709909054470058064a'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:36.294162 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-72cffdc5-d5b5-48f9-a3e8-fa1fbf1995b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06dc59a28a4a4709909054470058064a'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:36.307575 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-72cffdc5-d5b5-48f9-a3e8-fa1fbf1995b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06dc59a28a4a4709909054470058064a'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:36.315920 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-72cffdc5-d5b5-48f9-a3e8-fa1fbf1995b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06dc59a28a4a4709909054470058064a'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:36.316616 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-72cffdc5-d5b5-48f9-a3e8-fa1fbf1995b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06dc59a28a4a4709909054470058064a'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:36.322803 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1334/2655] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:28:36 2022] DELETE /identity/v3/users/06dc59a28a4a4709909054470058064a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:36.327543 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec8f42-cd96-4864-8f9f-aa7830782b99 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:36.328092 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec8f42-cd96-4864-8f9f-aa7830782b99 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:36.328356 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec8f42-cd96-4864-8f9f-aa7830782b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:36.345731 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-1fec8f42-cd96-4864-8f9f-aa7830782b99 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:36.346921 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1fec8f42-cd96-4864-8f9f-aa7830782b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:36.404586 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1322/2656] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:36.414454 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cffdc5-d5b5-48f9-a3e8-fa1fbf1995b1 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:36.415942 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-72cffdc5-d5b5-48f9-a3e8-fa1fbf1995b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:36.474143 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cffdc5-d5b5-48f9-a3e8-fa1fbf1995b1 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:36.477775 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fbf3e5-33f1-40ff-bc91-0ca0a00b49f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:36.478419 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fbf3e5-33f1-40ff-bc91-0ca0a00b49f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:36.478704 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fbf3e5-33f1-40ff-bc91-0ca0a00b49f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:36.478984 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fbf3e5-33f1-40ff-bc91-0ca0a00b49f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:36.488497 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fbf3e5-33f1-40ff-bc91-0ca0a00b49f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:36.489905 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fbf3e5-33f1-40ff-bc91-0ca0a00b49f1 service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:36.499754 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1335/2657] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:28:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:28:36.699125 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec8f42-cd96-4864-8f9f-aa7830782b99 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:36.700624 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1fec8f42-cd96-4864-8f9f-aa7830782b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:36.712552 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec8f42-cd96-4864-8f9f-aa7830782b99 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:36.717763 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def5ebd3-e8af-4ad5-8483-e562f34dccca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:36.718639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5ebd3-e8af-4ad5-8483-e562f34dccca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:36.718919 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5ebd3-e8af-4ad5-8483-e562f34dccca admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:36.719727 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5ebd3-e8af-4ad5-8483-e562f34dccca admin admin] PATH_INFO: `/v3/users/28de27ba3a664986bbb18cf2a1e4bced` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:36.720948 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def5ebd3-e8af-4ad5-8483-e562f34dccca admin admin] RBAC: Authorizing `identity:delete_user(user_id=28de27ba3a664986bbb18cf2a1e4bced)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:36.722878 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def5ebd3-e8af-4ad5-8483-e562f34dccca admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:36.775734 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-def5ebd3-e8af-4ad5-8483-e562f34dccca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28de27ba3a664986bbb18cf2a1e4bced'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:36.777869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fbf3e5-33f1-40ff-bc91-0ca0a00b49f1 service neutron] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:36.779547 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-d6fbf3e5-33f1-40ff-bc91-0ca0a00b49f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:36.785052 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-def5ebd3-e8af-4ad5-8483-e562f34dccca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28de27ba3a664986bbb18cf2a1e4bced'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:36.796105 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fbf3e5-33f1-40ff-bc91-0ca0a00b49f1 service neutron] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:36.801016 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-def5ebd3-e8af-4ad5-8483-e562f34dccca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28de27ba3a664986bbb18cf2a1e4bced'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:36.801879 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-def5ebd3-e8af-4ad5-8483-e562f34dccca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28de27ba3a664986bbb18cf2a1e4bced'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:36.802252 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf8d8c8-2c2c-43f5-9767-734778e711af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:36.803485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf8d8c8-2c2c-43f5-9767-734778e711af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:36.803915 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf8d8c8-2c2c-43f5-9767-734778e711af admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:36.804267 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf8d8c8-2c2c-43f5-9767-734778e711af admin admin] PATH_INFO: `/v3/projects/6b68fed95706473cb2c06c72558da3e2` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:36.805695 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf8d8c8-2c2c-43f5-9767-734778e711af admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b68fed95706473cb2c06c72558da3e2)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:36.807775 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf8d8c8-2c2c-43f5-9767-734778e711af admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:36.810565 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-def5ebd3-e8af-4ad5-8483-e562f34dccca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28de27ba3a664986bbb18cf2a1e4bced'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:36.817713 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1323/2658] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:28:36 2022] DELETE /identity/v3/users/28de27ba3a664986bbb18cf2a1e4bced => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:36.822530 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106dad1-0e57-4661-b31f-14941ef1407a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:36.822824 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106dad1-0e57-4661-b31f-14941ef1407a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:36.823107 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106dad1-0e57-4661-b31f-14941ef1407a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:36.829342 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:36.845148 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-6106dad1-0e57-4661-b31f-14941ef1407a None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:36.846465 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6106dad1-0e57-4661-b31f-14941ef1407a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:36.859267 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-3cf8d8c8-2c2c-43f5-9767-734778e711af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b68fed95706473cb2c06c72558da3e2', 'request_id': 'req-3cf8d8c8-2c2c-43f5-9767-734778e711af'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:36.868590 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-3cf8d8c8-2c2c-43f5-9767-734778e711af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b68fed95706473cb2c06c72558da3e2', 'request_id': 'req-3cf8d8c8-2c2c-43f5-9767-734778e711af'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:36.880130 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-3cf8d8c8-2c2c-43f5-9767-734778e711af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b68fed95706473cb2c06c72558da3e2', 'request_id': 'req-3cf8d8c8-2c2c-43f5-9767-734778e711af'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:36.898901 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-3cf8d8c8-2c2c-43f5-9767-734778e711af admin admin] The token cache is being invalidate because project 6b68fed95706473cb2c06c72558da3e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:28:36.899193 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-3cf8d8c8-2c2c-43f5-9767-734778e711af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:36.899949 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-3cf8d8c8-2c2c-43f5-9767-734778e711af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:36.902854 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1336/2659] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:28:36 2022] DELETE /identity/v3/projects/6b68fed95706473cb2c06c72558da3e2 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:36.907837 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a081479-ba02-467a-ac62-2ba857d9d495 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:36.908125 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a081479-ba02-467a-ac62-2ba857d9d495 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:36.908401 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a081479-ba02-467a-ac62-2ba857d9d495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:36.915735 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1324/2660] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:28:36.926825 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6106dad1-0e57-4661-b31f-14941ef1407a None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:36.928211 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6106dad1-0e57-4661-b31f-14941ef1407a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:36.931239 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-6a081479-ba02-467a-ac62-2ba857d9d495 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:36.932661 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-6a081479-ba02-467a-ac62-2ba857d9d495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:36.948509 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1337/2661] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:28:36.950055 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf246b7-a02d-4da3-8037-ac69419658c5 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:36.950350 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf246b7-a02d-4da3-8037-ac69419658c5 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:36.950662 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf246b7-a02d-4da3-8037-ac69419658c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:36.974312 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-fbf246b7-a02d-4da3-8037-ac69419658c5 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:36.975593 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-fbf246b7-a02d-4da3-8037-ac69419658c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:36.985670 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1338/2662] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:36.990225 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf246b7-a02d-4da3-8037-ac69419658c5 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:37.000408 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf246b7-a02d-4da3-8037-ac69419658c5 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:37.003087 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6106dad1-0e57-4661-b31f-14941ef1407a None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:37.005703 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824ad71-aa30-4981-b711-c9df68825eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:37.006384 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824ad71-aa30-4981-b711-c9df68825eb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:37.006706 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824ad71-aa30-4981-b711-c9df68825eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:37.006945 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824ad71-aa30-4981-b711-c9df68825eb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:37.007599 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f3ae0a-2ae1-4497-8415-1bffc32ef1d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:37.007733 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8824ad71-aa30-4981-b711-c9df68825eb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:28:37.008076 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8824ad71-aa30-4981-b711-c9df68825eb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:37.008400 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3ae0a-2ae1-4497-8415-1bffc32ef1d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:37.008678 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3ae0a-2ae1-4497-8415-1bffc32ef1d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:37.008957 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3ae0a-2ae1-4497-8415-1bffc32ef1d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:37.009789 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8824ad71-aa30-4981-b711-c9df68825eb1 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:37.011781 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:37.027934 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1339/2663] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:28:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:28:37.029252 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824ad71-aa30-4981-b711-c9df68825eb1 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:37.030125 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f3ae0a-2ae1-4497-8415-1bffc32ef1d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:37.031686 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f3ae0a-2ae1-4497-8415-1bffc32ef1d0 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:37.041126 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824ad71-aa30-4981-b711-c9df68825eb1 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:37.043690 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1325/2664] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:28:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 16:28:37.046870 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14800640-7c2a-4492-8aaf-692995d9773b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:37.048518 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14800640-7c2a-4492-8aaf-692995d9773b admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:37.049001 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14800640-7c2a-4492-8aaf-692995d9773b admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:37.049480 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14800640-7c2a-4492-8aaf-692995d9773b admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:37.050354 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14800640-7c2a-4492-8aaf-692995d9773b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:37.052523 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14800640-7c2a-4492-8aaf-692995d9773b admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:37.085349 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1340/2665] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:28:37 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:28:37.089561 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f3ae0a-2ae1-4497-8415-1bffc32ef1d0 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:37.102119 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f3ae0a-2ae1-4497-8415-1bffc32ef1d0 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:37.176836 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c608d4d-cb91-4301-afee-b699f51f7d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:37.177424 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c608d4d-cb91-4301-afee-b699f51f7d5e admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:37.177847 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c608d4d-cb91-4301-afee-b699f51f7d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:37.178243 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c608d4d-cb91-4301-afee-b699f51f7d5e admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:37.179557 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c608d4d-cb91-4301-afee-b699f51f7d5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:37.181581 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c608d4d-cb91-4301-afee-b699f51f7d5e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:37.205617 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1326/2666] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:28:37 2022] POST /identity/v3/users => generated 290 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:28:37.209773 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14800640-7c2a-4492-8aaf-692995d9773b admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:37.221455 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14800640-7c2a-4492-8aaf-692995d9773b admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:37.229958 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb706b-0467-4a62-98a6-a3afd0b63b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:37.230648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb706b-0467-4a62-98a6-a3afd0b63b91 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:37.230932 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb706b-0467-4a62-98a6-a3afd0b63b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:37.231208 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb706b-0467-4a62-98a6-a3afd0b63b91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:37.232111 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fb706b-0467-4a62-98a6-a3afd0b63b91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:37.233718 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fb706b-0467-4a62-98a6-a3afd0b63b91 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:37.244249 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1341/2667] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:28:37.248337 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c608d4d-cb91-4301-afee-b699f51f7d5e admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:37.259390 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c608d4d-cb91-4301-afee-b699f51f7d5e admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:37.264198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8028451c-92bd-4acd-a9c7-c2a9bdd1a8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:37.264860 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8028451c-92bd-4acd-a9c7-c2a9bdd1a8ec admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:37.265146 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8028451c-92bd-4acd-a9c7-c2a9bdd1a8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:37.265426 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8028451c-92bd-4acd-a9c7-c2a9bdd1a8ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:37.266736 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8028451c-92bd-4acd-a9c7-c2a9bdd1a8ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:37.268376 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8028451c-92bd-4acd-a9c7-c2a9bdd1a8ec admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:37.278963 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1327/2668] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:28:37.283678 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb706b-0467-4a62-98a6-a3afd0b63b91 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:37.293967 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb706b-0467-4a62-98a6-a3afd0b63b91 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:37.298697 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822dbcbf-1510-44f9-8104-5b8953bbbfc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:37.299678 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822dbcbf-1510-44f9-8104-5b8953bbbfc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:37.299966 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822dbcbf-1510-44f9-8104-5b8953bbbfc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:37.300246 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822dbcbf-1510-44f9-8104-5b8953bbbfc6 admin admin] PATH_INFO: `/v3/projects/036a7f012aa34ff18cba47655550b87d/users/4e4f66dc019b48799109dced0e62e528/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:37.314892 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822dbcbf-1510-44f9-8104-5b8953bbbfc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=036a7f012aa34ff18cba47655550b87d, user_id=4e4f66dc019b48799109dced0e62e528, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:37.317565 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822dbcbf-1510-44f9-8104-5b8953bbbfc6 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:37.333049 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1342/2669] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:28:37 2022] PUT /identity/v3/projects/036a7f012aa34ff18cba47655550b87d/users/4e4f66dc019b48799109dced0e62e528/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:37.346525 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f2e4c-1056-4006-aeec-33c28ea93ad8 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:37.347156 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f2e4c-1056-4006-aeec-33c28ea93ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:37.347156 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f2e4c-1056-4006-aeec-33c28ea93ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:37.348975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822dbcbf-1510-44f9-8104-5b8953bbbfc6 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:37.351333 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-822dbcbf-1510-44f9-8104-5b8953bbbfc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:37.379546 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-765f2e4c-1056-4006-aeec-33c28ea93ad8 None None] MFA Rules not processed for user `4e4f66dc019b48799109dced0e62e528`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:37.381712 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-765f2e4c-1056-4006-aeec-33c28ea93ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:37.396163 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:37.428867 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822dbcbf-1510-44f9-8104-5b8953bbbfc6 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:37.433979 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a2974c-c7a3-4851-8435-c11facc83ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:37.434845 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2974c-c7a3-4851-8435-c11facc83ef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:37.435179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2974c-c7a3-4851-8435-c11facc83ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:37.435443 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2974c-c7a3-4851-8435-c11facc83ef3 admin admin] PATH_INFO: `/v3/projects/caa595780261498997d276a7269332cd` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:37.436655 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a2974c-c7a3-4851-8435-c11facc83ef3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=caa595780261498997d276a7269332cd)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:37.438591 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a2974c-c7a3-4851-8435-c11facc83ef3 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:37.452351 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1328/2670] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:28:37.457283 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f2e4c-1056-4006-aeec-33c28ea93ad8 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:37.467928 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f2e4c-1056-4006-aeec-33c28ea93ad8 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:37.473371 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7215ee4-6956-43d4-8e0b-d55dd4d30bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:37.474018 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7215ee4-6956-43d4-8e0b-d55dd4d30bde admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:37.474307 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7215ee4-6956-43d4-8e0b-d55dd4d30bde admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:37.474590 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7215ee4-6956-43d4-8e0b-d55dd4d30bde admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:37.475322 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7215ee4-6956-43d4-8e0b-d55dd4d30bde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:37.477192 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7215ee4-6956-43d4-8e0b-d55dd4d30bde admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:37.487064 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-f9a2974c-c7a3-4851-8435-c11facc83ef3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'caa595780261498997d276a7269332cd', 'request_id': 'req-f9a2974c-c7a3-4851-8435-c11facc83ef3'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:37.494472 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-f9a2974c-c7a3-4851-8435-c11facc83ef3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'caa595780261498997d276a7269332cd', 'request_id': 'req-f9a2974c-c7a3-4851-8435-c11facc83ef3'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:37.502613 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-f9a2974c-c7a3-4851-8435-c11facc83ef3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'caa595780261498997d276a7269332cd', 'request_id': 'req-f9a2974c-c7a3-4851-8435-c11facc83ef3'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:37.506501 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1329/2671] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:28:37 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:28:37.507556 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7215ee4-6956-43d4-8e0b-d55dd4d30bde admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:37.514283 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-f9a2974c-c7a3-4851-8435-c11facc83ef3 admin admin] The token cache is being invalidate because project caa595780261498997d276a7269332cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:28:37.514701 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-f9a2974c-c7a3-4851-8435-c11facc83ef3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:37.515444 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-f9a2974c-c7a3-4851-8435-c11facc83ef3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:37.517623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1343/2672] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:28:37 2022] DELETE /identity/v3/projects/caa595780261498997d276a7269332cd => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:28:37.522506 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7314be8-ffc3-4426-b3b2-b0c6efda192f None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:37.522738 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7314be8-ffc3-4426-b3b2-b0c6efda192f None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:37.523019 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7314be8-ffc3-4426-b3b2-b0c6efda192f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:37.541218 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-c7314be8-ffc3-4426-b3b2-b0c6efda192f None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:37.542435 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-c7314be8-ffc3-4426-b3b2-b0c6efda192f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:37.562496 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:37.572078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7215ee4-6956-43d4-8e0b-d55dd4d30bde admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:37.577336 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc363285-2427-40ac-be74-523016f07ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:37.578034 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc363285-2427-40ac-be74-523016f07ec0 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:37.578323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc363285-2427-40ac-be74-523016f07ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:37.578655 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc363285-2427-40ac-be74-523016f07ec0 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:37.579981 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc363285-2427-40ac-be74-523016f07ec0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:37.582218 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc363285-2427-40ac-be74-523016f07ec0 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:37.603796 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1330/2673] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:28:37 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:28:37.604821 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1344/2674] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:28:37.607493 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc363285-2427-40ac-be74-523016f07ec0 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:37.609129 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-dc363285-2427-40ac-be74-523016f07ec0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:37.612157 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01988d-5b75-4dd4-8584-8e40abd160eb None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:37.612451 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01988d-5b75-4dd4-8584-8e40abd160eb None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:37.612736 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01988d-5b75-4dd4-8584-8e40abd160eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:37.621088 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc363285-2427-40ac-be74-523016f07ec0 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:37.626389 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62097c-a9ee-438b-90c6-98a5de34fb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:37.627366 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62097c-a9ee-438b-90c6-98a5de34fb2f admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:37.627762 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62097c-a9ee-438b-90c6-98a5de34fb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:37.628047 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62097c-a9ee-438b-90c6-98a5de34fb2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:37.628783 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b62097c-a9ee-438b-90c6-98a5de34fb2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:37.630395 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b62097c-a9ee-438b-90c6-98a5de34fb2f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:37.632603 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-bb01988d-5b75-4dd4-8584-8e40abd160eb None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:37.633836 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-bb01988d-5b75-4dd4-8584-8e40abd160eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:37.641050 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1331/2675] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:28:37.642806 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1345/2676] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:37.646078 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62097c-a9ee-438b-90c6-98a5de34fb2f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:37.647006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb01988d-5b75-4dd4-8584-8e40abd160eb None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:37.656048 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62097c-a9ee-438b-90c6-98a5de34fb2f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:37.656189 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb01988d-5b75-4dd4-8584-8e40abd160eb None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:37.660590 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911eb83-9d3b-40af-9e76-497a961a5163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:37.660750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f48e8-ef71-4e3a-a759-4a3c0c55b795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:37.661278 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f48e8-ef71-4e3a-a759-4a3c0c55b795 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:37.661446 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911eb83-9d3b-40af-9e76-497a961a5163 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:37.661592 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f48e8-ef71-4e3a-a759-4a3c0c55b795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:37.661730 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911eb83-9d3b-40af-9e76-497a961a5163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:37.661868 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f48e8-ef71-4e3a-a759-4a3c0c55b795 admin admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:37.662007 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911eb83-9d3b-40af-9e76-497a961a5163 admin admin] PATH_INFO: `/v3/projects/5a857bdc4a924122b1a8cf32b9d0b64c/users/1620b861f00842e1b0ad90ef84095e4d/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:37.662586 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024f48e8-ef71-4e3a-a759-4a3c0c55b795 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:28:37.662900 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024f48e8-ef71-4e3a-a759-4a3c0c55b795 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:37.664526 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024f48e8-ef71-4e3a-a759-4a3c0c55b795 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:37.676116 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1346/2677] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:28:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:28:37.676612 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6911eb83-9d3b-40af-9e76-497a961a5163 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a857bdc4a924122b1a8cf32b9d0b64c, user_id=1620b861f00842e1b0ad90ef84095e4d, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:37.679321 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6911eb83-9d3b-40af-9e76-497a961a5163 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:37.680349 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f48e8-ef71-4e3a-a759-4a3c0c55b795 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:37.690595 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f48e8-ef71-4e3a-a759-4a3c0c55b795 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:37.693252 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1332/2678] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:28:37 2022] PUT /identity/v3/projects/5a857bdc4a924122b1a8cf32b9d0b64c/users/1620b861f00842e1b0ad90ef84095e4d/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:28:37.697260 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911eb83-9d3b-40af-9e76-497a961a5163 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:37.744105 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:37.748691 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07298794-84f3-4a91-b710-19b1835e88a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:37.749431 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07298794-84f3-4a91-b710-19b1835e88a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:37.749911 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07298794-84f3-4a91-b710-19b1835e88a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:37.750301 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07298794-84f3-4a91-b710-19b1835e88a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:37.751652 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07298794-84f3-4a91-b710-19b1835e88a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:37.753575 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07298794-84f3-4a91-b710-19b1835e88a1 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:37.759121 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911eb83-9d3b-40af-9e76-497a961a5163 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:37.763753 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823df50-c064-4875-85ec-822765797eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:37.764535 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823df50-c064-4875-85ec-822765797eef admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:37.764820 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823df50-c064-4875-85ec-822765797eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:37.765096 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823df50-c064-4875-85ec-822765797eef admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:37.765815 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823df50-c064-4875-85ec-822765797eef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:37.767975 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823df50-c064-4875-85ec-822765797eef admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:37.778411 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1333/2679] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:28:37.783493 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823df50-c064-4875-85ec-822765797eef admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:37.784889 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1347/2680] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:28:37 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:28:37.788940 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07298794-84f3-4a91-b710-19b1835e88a1 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:37.793270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823df50-c064-4875-85ec-822765797eef admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:37.849573 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b456988-4946-44c0-a36a-a7045cdb8049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:37.850451 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b456988-4946-44c0-a36a-a7045cdb8049 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:37.850734 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b456988-4946-44c0-a36a-a7045cdb8049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:37.851021 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b456988-4946-44c0-a36a-a7045cdb8049 admin admin] PATH_INFO: `/v3/projects/5a857bdc4a924122b1a8cf32b9d0b64c/users/1620b861f00842e1b0ad90ef84095e4d/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:37.853005 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b456988-4946-44c0-a36a-a7045cdb8049 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a857bdc4a924122b1a8cf32b9d0b64c, user_id=1620b861f00842e1b0ad90ef84095e4d, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:37.853674 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07298794-84f3-4a91-b710-19b1835e88a1 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:37.855674 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b456988-4946-44c0-a36a-a7045cdb8049 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:37.859142 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ab151-38e7-45a1-b4b4-d4708f0d114d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:37.860245 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ab151-38e7-45a1-b4b4-d4708f0d114d admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:37.860658 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ab151-38e7-45a1-b4b4-d4708f0d114d admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:37.861115 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ab151-38e7-45a1-b4b4-d4708f0d114d admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:37.862577 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0ab151-38e7-45a1-b4b4-d4708f0d114d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:37.865472 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0ab151-38e7-45a1-b4b4-d4708f0d114d admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:37.865844 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1334/2681] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:28:37 2022] PUT /identity/v3/projects/5a857bdc4a924122b1a8cf32b9d0b64c/users/1620b861f00842e1b0ad90ef84095e4d/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:28:37.875747 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce46990e-b8aa-4865-a34a-7b9df2b3fdac None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:37.876111 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce46990e-b8aa-4865-a34a-7b9df2b3fdac None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:37.876574 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce46990e-b8aa-4865-a34a-7b9df2b3fdac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:37.889943 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1348/2682] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:28:37 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:28:37.891656 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ab151-38e7-45a1-b4b4-d4708f0d114d admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:37.901490 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ab151-38e7-45a1-b4b4-d4708f0d114d admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:37.904077 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-ce46990e-b8aa-4865-a34a-7b9df2b3fdac None None] MFA Rules not processed for user `1620b861f00842e1b0ad90ef84095e4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:37.905279 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ce46990e-b8aa-4865-a34a-7b9df2b3fdac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:37.920941 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:37.960818 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cdaefc-dad4-4099-9517-736d9104309e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:37.961420 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cdaefc-dad4-4099-9517-736d9104309e admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:37.961708 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cdaefc-dad4-4099-9517-736d9104309e admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:37.961982 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cdaefc-dad4-4099-9517-736d9104309e admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:37.962701 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cdaefc-dad4-4099-9517-736d9104309e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:37.964509 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cdaefc-dad4-4099-9517-736d9104309e admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:37.973238 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1335/2683] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:37 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:28:37.974893 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1349/2684] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:28:37.978759 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce46990e-b8aa-4865-a34a-7b9df2b3fdac None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:37.985653 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cdaefc-dad4-4099-9517-736d9104309e admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:37.987363 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-59cdaefc-dad4-4099-9517-736d9104309e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:37.989532 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce46990e-b8aa-4865-a34a-7b9df2b3fdac None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:37.994060 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d72e34-aa10-4e06-865e-64bf30c913fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:37.994707 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d72e34-aa10-4e06-865e-64bf30c913fe admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:37.994990 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d72e34-aa10-4e06-865e-64bf30c913fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:37.995260 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d72e34-aa10-4e06-865e-64bf30c913fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:37.996020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d72e34-aa10-4e06-865e-64bf30c913fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:37.997598 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d72e34-aa10-4e06-865e-64bf30c913fe admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:38.007451 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1336/2685] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:28:38.011490 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d72e34-aa10-4e06-865e-64bf30c913fe admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:38.023430 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d72e34-aa10-4e06-865e-64bf30c913fe admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:38.028486 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c52eb-933a-4cda-b4bf-717474d7f7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:38.029347 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c52eb-933a-4cda-b4bf-717474d7f7d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:38.029633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c52eb-933a-4cda-b4bf-717474d7f7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:38.029911 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c52eb-933a-4cda-b4bf-717474d7f7d6 admin admin] PATH_INFO: `/v3/projects/0ba60488b1ba436fb10642b6372a2648/users/83dcfa0ebeec4efc8a7d0b4c366f445d/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:38.043819 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cdaefc-dad4-4099-9517-736d9104309e admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:38.045571 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59c52eb-933a-4cda-b4bf-717474d7f7d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ba60488b1ba436fb10642b6372a2648, user_id=83dcfa0ebeec4efc8a7d0b4c366f445d, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:38.047470 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef6d16-c658-45e7-8a56-3211cc9fe063 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:38.048113 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef6d16-c658-45e7-8a56-3211cc9fe063 service cinder] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:38.048305 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59c52eb-933a-4cda-b4bf-717474d7f7d6 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:38.048431 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef6d16-c658-45e7-8a56-3211cc9fe063 service cinder] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:38.048658 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef6d16-c658-45e7-8a56-3211cc9fe063 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:38.058187 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef6d16-c658-45e7-8a56-3211cc9fe063 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:38.059724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef6d16-c658-45e7-8a56-3211cc9fe063 service cinder] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:38.061266 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1337/2686] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:28:38 2022] PUT /identity/v3/projects/0ba60488b1ba436fb10642b6372a2648/users/83dcfa0ebeec4efc8a7d0b4c366f445d/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:28:38.069469 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1350/2687] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:28:37 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 16:28:38.071333 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4de618-8641-4b28-88f0-13957b979f03 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:38.071640 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4de618-8641-4b28-88f0-13957b979f03 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:38.071925 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4de618-8641-4b28-88f0-13957b979f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:38.099926 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-da4de618-8641-4b28-88f0-13957b979f03 None None] MFA Rules not processed for user `83dcfa0ebeec4efc8a7d0b4c366f445d`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:38.101156 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-da4de618-8641-4b28-88f0-13957b979f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:38.114680 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:38.155311 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef6d16-c658-45e7-8a56-3211cc9fe063 service cinder] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:38.164467 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef6d16-c658-45e7-8a56-3211cc9fe063 service cinder] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:38.173142 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1338/2688] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:38.183821 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4de618-8641-4b28-88f0-13957b979f03 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:38.185213 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-da4de618-8641-4b28-88f0-13957b979f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:38.216754 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd284895-be74-4902-9be3-624a8d048b15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:38.217310 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd284895-be74-4902-9be3-624a8d048b15 service cinder] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:38.217605 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd284895-be74-4902-9be3-624a8d048b15 service cinder] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:38.217860 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd284895-be74-4902-9be3-624a8d048b15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:38.220322 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-cd284895-be74-4902-9be3-624a8d048b15 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:38.244280 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4de618-8641-4b28-88f0-13957b979f03 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:38.248339 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb85e1a-6e00-4979-8c8a-35f0d2e8ef6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:38.248984 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb85e1a-6e00-4979-8c8a-35f0d2e8ef6c service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:38.249274 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb85e1a-6e00-4979-8c8a-35f0d2e8ef6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:38.249556 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb85e1a-6e00-4979-8c8a-35f0d2e8ef6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:38.258888 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb85e1a-6e00-4979-8c8a-35f0d2e8ef6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:38.260533 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb85e1a-6e00-4979-8c8a-35f0d2e8ef6c service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:38.270654 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1339/2689] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:28:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 16:28:38.282629 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd284895-be74-4902-9be3-624a8d048b15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:38.284166 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd284895-be74-4902-9be3-624a8d048b15 service cinder] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:38.293266 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1351/2690] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:28:38 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:28:48.265737 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=46 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:48.288936 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=93 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:49.121582 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb85e1a-6e00-4979-8c8a-35f0d2e8ef6c service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:49.123248 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1bb85e1a-6e00-4979-8c8a-35f0d2e8ef6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:49.180790 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb85e1a-6e00-4979-8c8a-35f0d2e8ef6c service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:49.184473 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b45f2-3282-45c9-b432-18eeef4b5707 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:49.185109 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b45f2-3282-45c9-b432-18eeef4b5707 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:49.185495 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b45f2-3282-45c9-b432-18eeef4b5707 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:49.185755 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b45f2-3282-45c9-b432-18eeef4b5707 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:49.188625 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9f0b45f2-3282-45c9-b432-18eeef4b5707 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:49.276890 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0b45f2-3282-45c9-b432-18eeef4b5707 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:49.278352 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0b45f2-3282-45c9-b432-18eeef4b5707 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:49.297503 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1340/2691] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:28:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:28:50.149013 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd284895-be74-4902-9be3-624a8d048b15 service cinder] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:50.152257 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-cd284895-be74-4902-9be3-624a8d048b15 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:50.310023 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd284895-be74-4902-9be3-624a8d048b15 service cinder] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:50.317200 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff12be-04f1-4076-97f4-fe4a0aa696c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:50.317993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff12be-04f1-4076-97f4-fe4a0aa696c7 service nova] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:50.318423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff12be-04f1-4076-97f4-fe4a0aa696c7 service nova] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:50.318942 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff12be-04f1-4076-97f4-fe4a0aa696c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:50.321677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-e8ff12be-04f1-4076-97f4-fe4a0aa696c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:50.339531 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff12be-04f1-4076-97f4-fe4a0aa696c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:50.342230 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff12be-04f1-4076-97f4-fe4a0aa696c7 service nova] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:50.356254 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1352/2692] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:28:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:28:51.960236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b45f2-3282-45c9-b432-18eeef4b5707 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:52.067471 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b45f2-3282-45c9-b432-18eeef4b5707 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:52.081786 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0ec73-4134-475c-b9ae-a69fc432ee13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:52.082953 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0ec73-4134-475c-b9ae-a69fc432ee13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:52.083427 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0ec73-4134-475c-b9ae-a69fc432ee13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:52.083898 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0ec73-4134-475c-b9ae-a69fc432ee13 admin admin] PATH_INFO: `/v3/users/4e4f66dc019b48799109dced0e62e528` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:52.085146 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0ec73-4134-475c-b9ae-a69fc432ee13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e4f66dc019b48799109dced0e62e528)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:52.087229 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0ec73-4134-475c-b9ae-a69fc432ee13 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:52.130112 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:52.159151 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2ad0ec73-4134-475c-b9ae-a69fc432ee13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e4f66dc019b48799109dced0e62e528'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:52.168619 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2ad0ec73-4134-475c-b9ae-a69fc432ee13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e4f66dc019b48799109dced0e62e528'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:52.185944 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2ad0ec73-4134-475c-b9ae-a69fc432ee13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e4f66dc019b48799109dced0e62e528'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:52.186719 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2ad0ec73-4134-475c-b9ae-a69fc432ee13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e4f66dc019b48799109dced0e62e528'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:52.196563 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2ad0ec73-4134-475c-b9ae-a69fc432ee13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e4f66dc019b48799109dced0e62e528'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:52.203181 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1341/2693] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:28:51 2022] DELETE /identity/v3/users/4e4f66dc019b48799109dced0e62e528 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:52.208599 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d3577-6ef9-49fb-b896-25cddbbd05b2 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:52.209430 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d3577-6ef9-49fb-b896-25cddbbd05b2 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:52.209720 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d3577-6ef9-49fb-b896-25cddbbd05b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:52.229833 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-134d3577-6ef9-49fb-b896-25cddbbd05b2 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:52.231401 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-134d3577-6ef9-49fb-b896-25cddbbd05b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:52.330396 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1353/2694] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:52.340337 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0ec73-4134-475c-b9ae-a69fc432ee13 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:52.341814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2ad0ec73-4134-475c-b9ae-a69fc432ee13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:52.404435 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0ec73-4134-475c-b9ae-a69fc432ee13 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:52.408362 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fc827b-1240-498e-a465-18ac898822d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:52.409129 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc827b-1240-498e-a465-18ac898822d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:52.409482 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc827b-1240-498e-a465-18ac898822d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:52.409840 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc827b-1240-498e-a465-18ac898822d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:52.419462 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fc827b-1240-498e-a465-18ac898822d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:52.420973 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fc827b-1240-498e-a465-18ac898822d4 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:52.431733 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1342/2695] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:28:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:28:52.729994 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d3577-6ef9-49fb-b896-25cddbbd05b2 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:52.731532 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-134d3577-6ef9-49fb-b896-25cddbbd05b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:52.753823 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d3577-6ef9-49fb-b896-25cddbbd05b2 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:52.761840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ee80f-ef2f-4620-bc8a-001b04cc2158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:52.762564 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ee80f-ef2f-4620-bc8a-001b04cc2158 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:52.762922 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ee80f-ef2f-4620-bc8a-001b04cc2158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:52.775530 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ee80f-ef2f-4620-bc8a-001b04cc2158 admin admin] PATH_INFO: `/v3/projects/036a7f012aa34ff18cba47655550b87d` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:52.775530 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ee80f-ef2f-4620-bc8a-001b04cc2158 admin admin] RBAC: Authorizing `identity:delete_project(project_id=036a7f012aa34ff18cba47655550b87d)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:52.786151 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ee80f-ef2f-4620-bc8a-001b04cc2158 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:52.857370 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-5f3ee80f-ef2f-4620-bc8a-001b04cc2158 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '036a7f012aa34ff18cba47655550b87d', 'request_id': 'req-5f3ee80f-ef2f-4620-bc8a-001b04cc2158'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:52.864965 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-5f3ee80f-ef2f-4620-bc8a-001b04cc2158 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '036a7f012aa34ff18cba47655550b87d', 'request_id': 'req-5f3ee80f-ef2f-4620-bc8a-001b04cc2158'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:52.874266 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-5f3ee80f-ef2f-4620-bc8a-001b04cc2158 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '036a7f012aa34ff18cba47655550b87d', 'request_id': 'req-5f3ee80f-ef2f-4620-bc8a-001b04cc2158'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:52.894926 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-5f3ee80f-ef2f-4620-bc8a-001b04cc2158 admin admin] The token cache is being invalidate because project 036a7f012aa34ff18cba47655550b87d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:28:52.895228 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-5f3ee80f-ef2f-4620-bc8a-001b04cc2158 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:52.896120 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-5f3ee80f-ef2f-4620-bc8a-001b04cc2158 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:52.898546 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1354/2696] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:28:52 2022] DELETE /identity/v3/projects/036a7f012aa34ff18cba47655550b87d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:52.902842 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fc827b-1240-498e-a465-18ac898822d4 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:52.922036 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fc827b-1240-498e-a465-18ac898822d4 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:52.991664 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd0378-6f4c-4d06-a43e-e036abde85df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:52.992538 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd0378-6f4c-4d06-a43e-e036abde85df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:52.992824 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd0378-6f4c-4d06-a43e-e036abde85df admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:52.993104 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd0378-6f4c-4d06-a43e-e036abde85df admin admin] PATH_INFO: `/v3/users/1620b861f00842e1b0ad90ef84095e4d` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:52.994287 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcd0378-6f4c-4d06-a43e-e036abde85df admin admin] RBAC: Authorizing `identity:delete_user(user_id=1620b861f00842e1b0ad90ef84095e4d)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:52.996252 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcd0378-6f4c-4d06-a43e-e036abde85df admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:53.041337 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3fcd0378-6f4c-4d06-a43e-e036abde85df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1620b861f00842e1b0ad90ef84095e4d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:53.043741 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:53.049373 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3fcd0378-6f4c-4d06-a43e-e036abde85df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1620b861f00842e1b0ad90ef84095e4d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:53.062350 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3fcd0378-6f4c-4d06-a43e-e036abde85df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1620b861f00842e1b0ad90ef84095e4d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:53.062983 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3fcd0378-6f4c-4d06-a43e-e036abde85df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1620b861f00842e1b0ad90ef84095e4d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:53.069963 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3fcd0378-6f4c-4d06-a43e-e036abde85df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1620b861f00842e1b0ad90ef84095e4d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:53.080468 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1343/2697] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:28:52 2022] DELETE /identity/v3/users/1620b861f00842e1b0ad90ef84095e4d => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:53.240869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ee80f-ef2f-4620-bc8a-001b04cc2158 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:53.242330 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-5f3ee80f-ef2f-4620-bc8a-001b04cc2158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:53.259592 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:53.356374 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ee80f-ef2f-4620-bc8a-001b04cc2158 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:53.410105 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd0378-6f4c-4d06-a43e-e036abde85df admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:53.411511 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3fcd0378-6f4c-4d06-a43e-e036abde85df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:53.440122 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37fea6d-c17d-436a-be99-095fee789d3d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be901786f8934d559e0bac5d622f97cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:53.440920 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37fea6d-c17d-436a-be99-095fee789d3d service placement] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:53.441306 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37fea6d-c17d-436a-be99-095fee789d3d service placement] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:53.441698 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37fea6d-c17d-436a-be99-095fee789d3d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:53.443908 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-f37fea6d-c17d-436a-be99-095fee789d3d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:53.461001 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37fea6d-c17d-436a-be99-095fee789d3d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:53.462967 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37fea6d-c17d-436a-be99-095fee789d3d service placement] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:53.475174 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1355/2698] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:28:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:28:53.493132 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd0378-6f4c-4d06-a43e-e036abde85df admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:53.498026 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f05f783-ed8e-448d-bb04-5f15b7f97218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:53.498889 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05f783-ed8e-448d-bb04-5f15b7f97218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:53.499178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05f783-ed8e-448d-bb04-5f15b7f97218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:53.499489 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05f783-ed8e-448d-bb04-5f15b7f97218 admin admin] PATH_INFO: `/v3/projects/5a857bdc4a924122b1a8cf32b9d0b64c` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:53.500972 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f05f783-ed8e-448d-bb04-5f15b7f97218 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a857bdc4a924122b1a8cf32b9d0b64c)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:53.502969 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f05f783-ed8e-448d-bb04-5f15b7f97218 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:53.561539 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0f05f783-ed8e-448d-bb04-5f15b7f97218 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a857bdc4a924122b1a8cf32b9d0b64c', 'request_id': 'req-0f05f783-ed8e-448d-bb04-5f15b7f97218'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:53.568105 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0f05f783-ed8e-448d-bb04-5f15b7f97218 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a857bdc4a924122b1a8cf32b9d0b64c', 'request_id': 'req-0f05f783-ed8e-448d-bb04-5f15b7f97218'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:53.576522 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0f05f783-ed8e-448d-bb04-5f15b7f97218 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a857bdc4a924122b1a8cf32b9d0b64c', 'request_id': 'req-0f05f783-ed8e-448d-bb04-5f15b7f97218'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:53.588674 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0f05f783-ed8e-448d-bb04-5f15b7f97218 admin admin] The token cache is being invalidate because project 5a857bdc4a924122b1a8cf32b9d0b64c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:28:53.589158 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0f05f783-ed8e-448d-bb04-5f15b7f97218 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:53.589993 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0f05f783-ed8e-448d-bb04-5f15b7f97218 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:53.591879 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1344/2699] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:28:53 2022] DELETE /identity/v3/projects/5a857bdc4a924122b1a8cf32b9d0b64c => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:28:53.597880 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110443e-67b9-43cd-8a99-a986fc41267f None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:53.598178 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110443e-67b9-43cd-8a99-a986fc41267f None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:53.598463 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110443e-67b9-43cd-8a99-a986fc41267f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:53.616516 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-6110443e-67b9-43cd-8a99-a986fc41267f None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:53.617848 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-6110443e-67b9-43cd-8a99-a986fc41267f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:53.694430 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1356/2700] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:28:53.703726 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3a97c0-b941-4731-b7eb-cf20a58928b3 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:53.704035 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3a97c0-b941-4731-b7eb-cf20a58928b3 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:53.704348 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3a97c0-b941-4731-b7eb-cf20a58928b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:53.726372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-2f3a97c0-b941-4731-b7eb-cf20a58928b3 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:53.727636 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2f3a97c0-b941-4731-b7eb-cf20a58928b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:53.737956 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1345/2701] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:53.749984 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110443e-67b9-43cd-8a99-a986fc41267f None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:53.752869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110443e-67b9-43cd-8a99-a986fc41267f None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:53.757944 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea04144-0a12-46cb-9e59-82856af856d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:53.758611 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea04144-0a12-46cb-9e59-82856af856d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:53.758900 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea04144-0a12-46cb-9e59-82856af856d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:53.759177 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea04144-0a12-46cb-9e59-82856af856d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:53.760047 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea04144-0a12-46cb-9e59-82856af856d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:28:53.760360 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea04144-0a12-46cb-9e59-82856af856d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:53.761942 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea04144-0a12-46cb-9e59-82856af856d8 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:53.774072 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1357/2702] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:28:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:28:53.778499 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3a97c0-b941-4731-b7eb-cf20a58928b3 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:53.788182 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3a97c0-b941-4731-b7eb-cf20a58928b3 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:53.793486 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc22c559-0b87-46cc-a801-a4f370d94bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:53.793927 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22c559-0b87-46cc-a801-a4f370d94bb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:53.794218 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22c559-0b87-46cc-a801-a4f370d94bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:53.794503 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22c559-0b87-46cc-a801-a4f370d94bb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:53.795277 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc22c559-0b87-46cc-a801-a4f370d94bb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:53.797622 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc22c559-0b87-46cc-a801-a4f370d94bb4 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:53.829058 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1346/2703] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:28:53 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:28:53.833733 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea04144-0a12-46cb-9e59-82856af856d8 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:53.845945 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea04144-0a12-46cb-9e59-82856af856d8 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:53.851987 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=100 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:53.914737 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c76796-1332-46b4-b843-7b02f9dcd105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:53.915285 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c76796-1332-46b4-b843-7b02f9dcd105 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:53.915610 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c76796-1332-46b4-b843-7b02f9dcd105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:53.915891 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c76796-1332-46b4-b843-7b02f9dcd105 admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:53.916591 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c76796-1332-46b4-b843-7b02f9dcd105 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:53.918496 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c76796-1332-46b4-b843-7b02f9dcd105 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:53.956290 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1358/2704] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:28:53 2022] POST /identity/v3/users => generated 283 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:28:53.957307 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc22c559-0b87-46cc-a801-a4f370d94bb4 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:53.970657 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc22c559-0b87-46cc-a801-a4f370d94bb4 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:53.976633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b8ce9e-690b-4255-8913-272c061d6a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:53.977311 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b8ce9e-690b-4255-8913-272c061d6a7c admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:53.977606 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b8ce9e-690b-4255-8913-272c061d6a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:53.977991 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b8ce9e-690b-4255-8913-272c061d6a7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:53.978641 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b8ce9e-690b-4255-8913-272c061d6a7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:53.980771 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b8ce9e-690b-4255-8913-272c061d6a7c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:53.993862 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1347/2705] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:28:53.999013 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c76796-1332-46b4-b843-7b02f9dcd105 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:54.016410 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c76796-1332-46b4-b843-7b02f9dcd105 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:54.021977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcf5239-8a3b-4bcd-b8b0-19c8da979dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:54.022650 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf5239-8a3b-4bcd-b8b0-19c8da979dbf admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:54.022955 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf5239-8a3b-4bcd-b8b0-19c8da979dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:54.023216 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf5239-8a3b-4bcd-b8b0-19c8da979dbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:54.023984 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcf5239-8a3b-4bcd-b8b0-19c8da979dbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:54.025573 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcf5239-8a3b-4bcd-b8b0-19c8da979dbf admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:54.036228 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1359/2706] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:28:54.040528 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b8ce9e-690b-4255-8913-272c061d6a7c admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:54.051025 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b8ce9e-690b-4255-8913-272c061d6a7c admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:54.055854 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9196e9-f0d1-4124-be34-6258ce06a13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:54.056731 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9196e9-f0d1-4124-be34-6258ce06a13b admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:54.057018 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9196e9-f0d1-4124-be34-6258ce06a13b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:54.057296 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9196e9-f0d1-4124-be34-6258ce06a13b admin admin] PATH_INFO: `/v3/projects/3e567fee1e74497e8b00d1bc54f2b016/users/1d8d58a557e3472f8077ce8b269e3df7/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:54.072650 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9196e9-f0d1-4124-be34-6258ce06a13b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e567fee1e74497e8b00d1bc54f2b016, user_id=1d8d58a557e3472f8077ce8b269e3df7, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:54.075268 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9196e9-f0d1-4124-be34-6258ce06a13b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:54.077462 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:54.090309 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1348/2707] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:28:54 2022] PUT /identity/v3/projects/3e567fee1e74497e8b00d1bc54f2b016/users/1d8d58a557e3472f8077ce8b269e3df7/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:54.100457 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a2d46-9e46-4a58-90bf-d3c3297500a9 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:54.100754 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a2d46-9e46-4a58-90bf-d3c3297500a9 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:54.101041 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a2d46-9e46-4a58-90bf-d3c3297500a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:54.129404 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-d08a2d46-9e46-4a58-90bf-d3c3297500a9 None None] MFA Rules not processed for user `1d8d58a557e3472f8077ce8b269e3df7`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:54.130832 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-d08a2d46-9e46-4a58-90bf-d3c3297500a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:54.195538 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1360/2708] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:54 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:54.200166 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9196e9-f0d1-4124-be34-6258ce06a13b admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:54.263545 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9196e9-f0d1-4124-be34-6258ce06a13b admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:54.268360 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48681a5a-cdcf-4474-8a9d-93421a7c6c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:54.269019 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48681a5a-cdcf-4474-8a9d-93421a7c6c72 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:54.269305 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48681a5a-cdcf-4474-8a9d-93421a7c6c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:54.269585 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48681a5a-cdcf-4474-8a9d-93421a7c6c72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:54.270360 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48681a5a-cdcf-4474-8a9d-93421a7c6c72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:54.272511 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48681a5a-cdcf-4474-8a9d-93421a7c6c72 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:54.297586 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1349/2709] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:28:54 2022] POST /identity/v3/projects => generated 365 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:28:54.302284 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a2d46-9e46-4a58-90bf-d3c3297500a9 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:54.312937 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a2d46-9e46-4a58-90bf-d3c3297500a9 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:54.370748 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b137cb3-37b4-4028-9d99-3c0f180f1a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:54.371357 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b137cb3-37b4-4028-9d99-3c0f180f1a1a admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:54.371664 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b137cb3-37b4-4028-9d99-3c0f180f1a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:54.371946 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b137cb3-37b4-4028-9d99-3c0f180f1a1a admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:54.372641 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b137cb3-37b4-4028-9d99-3c0f180f1a1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:54.374643 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b137cb3-37b4-4028-9d99-3c0f180f1a1a admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:54.395522 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1361/2710] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:28:54 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:28:54.399518 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48681a5a-cdcf-4474-8a9d-93421a7c6c72 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:54.411174 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48681a5a-cdcf-4474-8a9d-93421a7c6c72 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:54.416504 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d6115-f157-4163-a680-20f61e78f953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:54.417362 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d6115-f157-4163-a680-20f61e78f953 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:54.417692 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d6115-f157-4163-a680-20f61e78f953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:54.418039 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d6115-f157-4163-a680-20f61e78f953 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:54.418804 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d6115-f157-4163-a680-20f61e78f953 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:54.420615 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d6115-f157-4163-a680-20f61e78f953 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:54.434530 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1350/2711] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:28:54.436390 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b137cb3-37b4-4028-9d99-3c0f180f1a1a admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:54.446430 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b137cb3-37b4-4028-9d99-3c0f180f1a1a admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:54.451081 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9b4386-5191-4e21-91ff-a1569f663d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:54.451994 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b4386-5191-4e21-91ff-a1569f663d50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:54.452280 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b4386-5191-4e21-91ff-a1569f663d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:54.452557 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b4386-5191-4e21-91ff-a1569f663d50 admin admin] PATH_INFO: `/v3/projects/6990531ed339437eb8b0326f647340f9/users/7cb9dc06430b4f768f25b5f1b1153dc0/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:54.454969 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:54.469201 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9b4386-5191-4e21-91ff-a1569f663d50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6990531ed339437eb8b0326f647340f9, user_id=7cb9dc06430b4f768f25b5f1b1153dc0, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:54.472464 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9b4386-5191-4e21-91ff-a1569f663d50 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:54.485263 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1362/2712] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:28:54 2022] PUT /identity/v3/projects/6990531ed339437eb8b0326f647340f9/users/7cb9dc06430b4f768f25b5f1b1153dc0/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:54.489722 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d6115-f157-4163-a680-20f61e78f953 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:54.499571 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d6115-f157-4163-a680-20f61e78f953 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:54.555633 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c40245-d367-4259-b1c2-7180e1c65b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:54.556476 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c40245-d367-4259-b1c2-7180e1c65b05 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:54.556851 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c40245-d367-4259-b1c2-7180e1c65b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:54.557258 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c40245-d367-4259-b1c2-7180e1c65b05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:54.557988 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c40245-d367-4259-b1c2-7180e1c65b05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:54.559618 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c40245-d367-4259-b1c2-7180e1c65b05 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:54.569281 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1351/2713] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:28:54.573326 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9b4386-5191-4e21-91ff-a1569f663d50 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:54.583554 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9b4386-5191-4e21-91ff-a1569f663d50 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:54.588164 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac723a35-b381-49ad-9ab7-4b1947e0b5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:54.589047 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac723a35-b381-49ad-9ab7-4b1947e0b5d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:54.589333 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac723a35-b381-49ad-9ab7-4b1947e0b5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:54.589611 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac723a35-b381-49ad-9ab7-4b1947e0b5d5 admin admin] PATH_INFO: `/v3/projects/6990531ed339437eb8b0326f647340f9/users/7cb9dc06430b4f768f25b5f1b1153dc0/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:54.591607 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac723a35-b381-49ad-9ab7-4b1947e0b5d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6990531ed339437eb8b0326f647340f9, user_id=7cb9dc06430b4f768f25b5f1b1153dc0, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:54.594214 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac723a35-b381-49ad-9ab7-4b1947e0b5d5 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:54.604770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1363/2714] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:28:54 2022] PUT /identity/v3/projects/6990531ed339437eb8b0326f647340f9/users/7cb9dc06430b4f768f25b5f1b1153dc0/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:54.615192 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea3334-d781-4a2b-a8b0-f4826d2e7e49 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:54.615526 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea3334-d781-4a2b-a8b0-f4826d2e7e49 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:54.615806 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea3334-d781-4a2b-a8b0-f4826d2e7e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:54.641625 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b0ea3334-d781-4a2b-a8b0-f4826d2e7e49 None None] MFA Rules not processed for user `7cb9dc06430b4f768f25b5f1b1153dc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:54.642837 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b0ea3334-d781-4a2b-a8b0-f4826d2e7e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:54.691726 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:54.711238 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1352/2715] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:54 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:54.722845 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac723a35-b381-49ad-9ab7-4b1947e0b5d5 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:54.724585 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-ac723a35-b381-49ad-9ab7-4b1947e0b5d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:54.805418 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac723a35-b381-49ad-9ab7-4b1947e0b5d5 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:54.809280 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37720da-b002-4783-a2e9-ff05d963c07e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:54.809927 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37720da-b002-4783-a2e9-ff05d963c07e service cinder] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:54.810304 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37720da-b002-4783-a2e9-ff05d963c07e service cinder] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:54.810471 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37720da-b002-4783-a2e9-ff05d963c07e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:54.819930 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37720da-b002-4783-a2e9-ff05d963c07e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:54.821456 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37720da-b002-4783-a2e9-ff05d963c07e service cinder] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:54.831599 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1364/2716] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:28:54 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:28:54.909120 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea3334-d781-4a2b-a8b0-f4826d2e7e49 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:54.919509 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea3334-d781-4a2b-a8b0-f4826d2e7e49 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:54.924093 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b212cd-600c-46a3-aca7-83bee57e6a3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:54.924791 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b212cd-600c-46a3-aca7-83bee57e6a3f service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:54.925079 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b212cd-600c-46a3-aca7-83bee57e6a3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:54.925355 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b212cd-600c-46a3-aca7-83bee57e6a3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:54.989417 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b212cd-600c-46a3-aca7-83bee57e6a3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:54.990930 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b212cd-600c-46a3-aca7-83bee57e6a3f service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:55.001637 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1353/2717] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:28:54 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:28:56.387133 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37720da-b002-4783-a2e9-ff05d963c07e service cinder] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:56.388692 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-d37720da-b002-4783-a2e9-ff05d963c07e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:56.466304 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37720da-b002-4783-a2e9-ff05d963c07e service cinder] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:56.473167 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130bd8c-5a68-4a06-a405-8756dca8de1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:56.474387 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130bd8c-5a68-4a06-a405-8756dca8de1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:56.474890 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130bd8c-5a68-4a06-a405-8756dca8de1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:56.475421 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130bd8c-5a68-4a06-a405-8756dca8de1c admin admin] PATH_INFO: `/v3/users/83dcfa0ebeec4efc8a7d0b4c366f445d` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:56.477283 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130bd8c-5a68-4a06-a405-8756dca8de1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=83dcfa0ebeec4efc8a7d0b4c366f445d)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:56.479385 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130bd8c-5a68-4a06-a405-8756dca8de1c admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:56.531600 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-f130bd8c-5a68-4a06-a405-8756dca8de1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83dcfa0ebeec4efc8a7d0b4c366f445d'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:56.539482 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-f130bd8c-5a68-4a06-a405-8756dca8de1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83dcfa0ebeec4efc8a7d0b4c366f445d'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:56.540770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:28:56.555712 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-f130bd8c-5a68-4a06-a405-8756dca8de1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83dcfa0ebeec4efc8a7d0b4c366f445d'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:56.562705 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-f130bd8c-5a68-4a06-a405-8756dca8de1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83dcfa0ebeec4efc8a7d0b4c366f445d'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:56.563430 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-f130bd8c-5a68-4a06-a405-8756dca8de1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83dcfa0ebeec4efc8a7d0b4c366f445d'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:56.569533 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1365/2718] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:28:56 2022] DELETE /identity/v3/users/83dcfa0ebeec4efc8a7d0b4c366f445d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:28:56.574362 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807f6a0-9ab9-4c16-a6bf-299cdb336dd1 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:56.574842 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807f6a0-9ab9-4c16-a6bf-299cdb336dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:56.574944 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807f6a0-9ab9-4c16-a6bf-299cdb336dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:56.596681 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-1807f6a0-9ab9-4c16-a6bf-299cdb336dd1 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:28:56.598246 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1807f6a0-9ab9-4c16-a6bf-299cdb336dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:56.676744 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1354/2719] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:28:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:28:56.687939 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130bd8c-5a68-4a06-a405-8756dca8de1c admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:56.688529 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-f130bd8c-5a68-4a06-a405-8756dca8de1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:56.747504 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130bd8c-5a68-4a06-a405-8756dca8de1c admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:56.750955 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4fb99f-dcbe-4e65-96dc-81e267d4ff9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:56.752215 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4fb99f-dcbe-4e65-96dc-81e267d4ff9f service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:56.752497 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4fb99f-dcbe-4e65-96dc-81e267d4ff9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:56.752771 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4fb99f-dcbe-4e65-96dc-81e267d4ff9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:56.761753 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4fb99f-dcbe-4e65-96dc-81e267d4ff9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:56.763207 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4fb99f-dcbe-4e65-96dc-81e267d4ff9f service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:56.773527 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1366/2720] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:28:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:28:57.039238 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1807f6a0-9ab9-4c16-a6bf-299cdb336dd1 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:28:57.040832 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1807f6a0-9ab9-4c16-a6bf-299cdb336dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:28:57.053666 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1807f6a0-9ab9-4c16-a6bf-299cdb336dd1 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:28:57.059542 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84bddba-06a4-4498-81ff-3437259ef536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:28:57.060422 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84bddba-06a4-4498-81ff-3437259ef536 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:28:57.060706 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84bddba-06a4-4498-81ff-3437259ef536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:28:57.061033 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84bddba-06a4-4498-81ff-3437259ef536 admin admin] PATH_INFO: `/v3/projects/0ba60488b1ba436fb10642b6372a2648` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:28:57.062234 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84bddba-06a4-4498-81ff-3437259ef536 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ba60488b1ba436fb10642b6372a2648)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:28:57.064756 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84bddba-06a4-4498-81ff-3437259ef536 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:28:57.109846 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a84bddba-06a4-4498-81ff-3437259ef536 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ba60488b1ba436fb10642b6372a2648', 'request_id': 'req-a84bddba-06a4-4498-81ff-3437259ef536'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:57.116919 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a84bddba-06a4-4498-81ff-3437259ef536 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ba60488b1ba436fb10642b6372a2648', 'request_id': 'req-a84bddba-06a4-4498-81ff-3437259ef536'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:57.123818 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a84bddba-06a4-4498-81ff-3437259ef536 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ba60488b1ba436fb10642b6372a2648', 'request_id': 'req-a84bddba-06a4-4498-81ff-3437259ef536'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:57.135608 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a84bddba-06a4-4498-81ff-3437259ef536 admin admin] The token cache is being invalidate because project 0ba60488b1ba436fb10642b6372a2648 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:28:57.135902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a84bddba-06a4-4498-81ff-3437259ef536 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:57.136539 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a84bddba-06a4-4498-81ff-3437259ef536 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:28:57.138091 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1355/2721] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:28:57 2022] DELETE /identity/v3/projects/0ba60488b1ba436fb10642b6372a2648 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:06.769222 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=37,keystone:DELETE=2 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:07.132595 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:14.117808 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4fb99f-dcbe-4e65-96dc-81e267d4ff9f service neutron] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:14.119249 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-2e4fb99f-dcbe-4e65-96dc-81e267d4ff9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:14.132037 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4fb99f-dcbe-4e65-96dc-81e267d4ff9f service neutron] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:14.188066 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8a15e-da25-45da-95bd-bfda8c21133e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:14.188901 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8a15e-da25-45da-95bd-bfda8c21133e service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:14.189185 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8a15e-da25-45da-95bd-bfda8c21133e service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:14.189460 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8a15e-da25-45da-95bd-bfda8c21133e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:14.192828 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-cba8a15e-da25-45da-95bd-bfda8c21133e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:14.263678 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba8a15e-da25-45da-95bd-bfda8c21133e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:14.265115 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba8a15e-da25-45da-95bd-bfda8c21133e service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:14.274845 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1367/2722] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:29:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:29:15.180825 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84bddba-06a4-4498-81ff-3437259ef536 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:15.182448 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a84bddba-06a4-4498-81ff-3437259ef536 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:15.196744 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84bddba-06a4-4498-81ff-3437259ef536 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:15.201641 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c9ea9-d876-4e3a-950f-1385b68c67ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:15.202531 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c9ea9-d876-4e3a-950f-1385b68c67ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:15.202819 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c9ea9-d876-4e3a-950f-1385b68c67ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:15.203093 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c9ea9-d876-4e3a-950f-1385b68c67ec admin admin] PATH_INFO: `/v3/users/1d8d58a557e3472f8077ce8b269e3df7` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:15.204917 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c9ea9-d876-4e3a-950f-1385b68c67ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d8d58a557e3472f8077ce8b269e3df7)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:15.207436 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c9ea9-d876-4e3a-950f-1385b68c67ec admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:15.292936 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-231c9ea9-d876-4e3a-950f-1385b68c67ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d8d58a557e3472f8077ce8b269e3df7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:15.303267 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-231c9ea9-d876-4e3a-950f-1385b68c67ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d8d58a557e3472f8077ce8b269e3df7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:15.328274 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-231c9ea9-d876-4e3a-950f-1385b68c67ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d8d58a557e3472f8077ce8b269e3df7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:15.329986 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-231c9ea9-d876-4e3a-950f-1385b68c67ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d8d58a557e3472f8077ce8b269e3df7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:15.339899 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-231c9ea9-d876-4e3a-950f-1385b68c67ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d8d58a557e3472f8077ce8b269e3df7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:15.348070 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1356/2723] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:29:15 2022] DELETE /identity/v3/users/1d8d58a557e3472f8077ce8b269e3df7 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:15.352088 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de6f3e-0657-4ce2-95b5-83f74b196d65 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:15.352574 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de6f3e-0657-4ce2-95b5-83f74b196d65 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:15.352843 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de6f3e-0657-4ce2-95b5-83f74b196d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:15.373094 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-79de6f3e-0657-4ce2-95b5-83f74b196d65 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:29:15.374229 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-79de6f3e-0657-4ce2-95b5-83f74b196d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:15.443681 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1368/2724] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:29:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:29:15.451525 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c9ea9-d876-4e3a-950f-1385b68c67ec admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:15.453295 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-231c9ea9-d876-4e3a-950f-1385b68c67ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:15.530713 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c9ea9-d876-4e3a-950f-1385b68c67ec admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:15.536779 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa87a4-adb3-406d-860e-329d1caa0611 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:15.538755 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa87a4-adb3-406d-860e-329d1caa0611 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:15.539265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa87a4-adb3-406d-860e-329d1caa0611 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:15.539798 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa87a4-adb3-406d-860e-329d1caa0611 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:15.557866 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfa87a4-adb3-406d-860e-329d1caa0611 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:15.560828 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfa87a4-adb3-406d-860e-329d1caa0611 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:15.580380 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1357/2725] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:29:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:29:15.968846 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79de6f3e-0657-4ce2-95b5-83f74b196d65 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:15.970471 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-79de6f3e-0657-4ce2-95b5-83f74b196d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:15.983889 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79de6f3e-0657-4ce2-95b5-83f74b196d65 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:15.990384 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e18b370-fcab-4a00-b97c-8279133e6f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:15.991389 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18b370-fcab-4a00-b97c-8279133e6f11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:15.991679 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18b370-fcab-4a00-b97c-8279133e6f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:15.992012 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18b370-fcab-4a00-b97c-8279133e6f11 admin admin] PATH_INFO: `/v3/projects/3e567fee1e74497e8b00d1bc54f2b016` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:15.993314 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e18b370-fcab-4a00-b97c-8279133e6f11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e567fee1e74497e8b00d1bc54f2b016)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:15.995284 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e18b370-fcab-4a00-b97c-8279133e6f11 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:16.054140 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:16.057643 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-5e18b370-fcab-4a00-b97c-8279133e6f11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e567fee1e74497e8b00d1bc54f2b016', 'request_id': 'req-5e18b370-fcab-4a00-b97c-8279133e6f11'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:16.067705 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-5e18b370-fcab-4a00-b97c-8279133e6f11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e567fee1e74497e8b00d1bc54f2b016', 'request_id': 'req-5e18b370-fcab-4a00-b97c-8279133e6f11'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:16.079054 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-5e18b370-fcab-4a00-b97c-8279133e6f11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e567fee1e74497e8b00d1bc54f2b016', 'request_id': 'req-5e18b370-fcab-4a00-b97c-8279133e6f11'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:16.090618 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-5e18b370-fcab-4a00-b97c-8279133e6f11 admin admin] The token cache is being invalidate because project 3e567fee1e74497e8b00d1bc54f2b016 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:29:16.090913 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-5e18b370-fcab-4a00-b97c-8279133e6f11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:16.091606 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-5e18b370-fcab-4a00-b97c-8279133e6f11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:16.093298 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1369/2726] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:29:15 2022] DELETE /identity/v3/projects/3e567fee1e74497e8b00d1bc54f2b016 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:16.098471 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa87a4-adb3-406d-860e-329d1caa0611 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:16.110662 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa87a4-adb3-406d-860e-329d1caa0611 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:16.178451 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262132b-668b-46cf-80c6-eb9b35ec89d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:16.179414 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262132b-668b-46cf-80c6-eb9b35ec89d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:16.179775 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262132b-668b-46cf-80c6-eb9b35ec89d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:16.180061 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262132b-668b-46cf-80c6-eb9b35ec89d2 admin admin] PATH_INFO: `/v3/users/7cb9dc06430b4f768f25b5f1b1153dc0` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:16.181256 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2262132b-668b-46cf-80c6-eb9b35ec89d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cb9dc06430b4f768f25b5f1b1153dc0)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:16.183192 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2262132b-668b-46cf-80c6-eb9b35ec89d2 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:16.201209 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:16.237973 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2262132b-668b-46cf-80c6-eb9b35ec89d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cb9dc06430b4f768f25b5f1b1153dc0'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:16.252767 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2262132b-668b-46cf-80c6-eb9b35ec89d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cb9dc06430b4f768f25b5f1b1153dc0'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:16.269890 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2262132b-668b-46cf-80c6-eb9b35ec89d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cb9dc06430b4f768f25b5f1b1153dc0'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:16.270690 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2262132b-668b-46cf-80c6-eb9b35ec89d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cb9dc06430b4f768f25b5f1b1153dc0'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:16.278100 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2262132b-668b-46cf-80c6-eb9b35ec89d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cb9dc06430b4f768f25b5f1b1153dc0'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:16.284808 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1358/2727] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:29:16 2022] DELETE /identity/v3/users/7cb9dc06430b4f768f25b5f1b1153dc0 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:16.592268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e18b370-fcab-4a00-b97c-8279133e6f11 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:16.593851 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-5e18b370-fcab-4a00-b97c-8279133e6f11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:16.609663 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e18b370-fcab-4a00-b97c-8279133e6f11 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:16.671665 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af1681-6bef-448e-a12c-0d9de28b87a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:16.672677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af1681-6bef-448e-a12c-0d9de28b87a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:16.672931 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af1681-6bef-448e-a12c-0d9de28b87a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:16.673211 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af1681-6bef-448e-a12c-0d9de28b87a6 admin admin] PATH_INFO: `/v3/projects/6990531ed339437eb8b0326f647340f9` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:16.674913 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af1681-6bef-448e-a12c-0d9de28b87a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6990531ed339437eb8b0326f647340f9)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:16.676854 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af1681-6bef-448e-a12c-0d9de28b87a6 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:16.752048 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-b6af1681-6bef-448e-a12c-0d9de28b87a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6990531ed339437eb8b0326f647340f9', 'request_id': 'req-b6af1681-6bef-448e-a12c-0d9de28b87a6'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:16.763980 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-b6af1681-6bef-448e-a12c-0d9de28b87a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6990531ed339437eb8b0326f647340f9', 'request_id': 'req-b6af1681-6bef-448e-a12c-0d9de28b87a6'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:16.775070 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-b6af1681-6bef-448e-a12c-0d9de28b87a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6990531ed339437eb8b0326f647340f9', 'request_id': 'req-b6af1681-6bef-448e-a12c-0d9de28b87a6'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:16.798561 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-b6af1681-6bef-448e-a12c-0d9de28b87a6 admin admin] The token cache is being invalidate because project 6990531ed339437eb8b0326f647340f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:29:16.799972 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-b6af1681-6bef-448e-a12c-0d9de28b87a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:16.800944 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-b6af1681-6bef-448e-a12c-0d9de28b87a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:16.803181 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1370/2728] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:29:16 2022] DELETE /identity/v3/projects/6990531ed339437eb8b0326f647340f9 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:16.807848 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e24aa-9957-4de8-81e4-92e112e2f535 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:16.808218 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e24aa-9957-4de8-81e4-92e112e2f535 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:16.808611 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e24aa-9957-4de8-81e4-92e112e2f535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:16.827226 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c50e24aa-9957-4de8-81e4-92e112e2f535 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:29:16.828622 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c50e24aa-9957-4de8-81e4-92e112e2f535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:16.919018 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1359/2729] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:29:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:29:16.922453 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df50dd-4624-41cb-9c2e-7b8df5894c85 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:16.922848 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df50dd-4624-41cb-9c2e-7b8df5894c85 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:16.923205 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df50dd-4624-41cb-9c2e-7b8df5894c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:16.943733 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-02df50dd-4624-41cb-9c2e-7b8df5894c85 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:29:16.944940 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-02df50dd-4624-41cb-9c2e-7b8df5894c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:16.955459 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1371/2730] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:29:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:29:16.960329 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e24aa-9957-4de8-81e4-92e112e2f535 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:16.972425 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e24aa-9957-4de8-81e4-92e112e2f535 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:16.978481 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b4693-71ec-4121-bf79-ffce65ba38be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:16.979163 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b4693-71ec-4121-bf79-ffce65ba38be admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:16.979627 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b4693-71ec-4121-bf79-ffce65ba38be admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:16.979838 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b4693-71ec-4121-bf79-ffce65ba38be admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:16.980564 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b4693-71ec-4121-bf79-ffce65ba38be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:29:16.980879 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b4693-71ec-4121-bf79-ffce65ba38be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:16.982523 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b4693-71ec-4121-bf79-ffce65ba38be admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:17.015714 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1360/2731] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:29:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:29:17.024860 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df50dd-4624-41cb-9c2e-7b8df5894c85 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:17.041011 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df50dd-4624-41cb-9c2e-7b8df5894c85 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:17.046709 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72b7f3-cbd8-4936-9d01-45ddb789ab09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:17.047392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72b7f3-cbd8-4936-9d01-45ddb789ab09 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:17.047888 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72b7f3-cbd8-4936-9d01-45ddb789ab09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:17.048176 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72b7f3-cbd8-4936-9d01-45ddb789ab09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:17.048882 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f72b7f3-cbd8-4936-9d01-45ddb789ab09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:17.052262 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f72b7f3-cbd8-4936-9d01-45ddb789ab09 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:17.089681 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1372/2732] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:29:17 2022] POST /identity/v3/projects => generated 383 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:29:17.095875 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b4693-71ec-4121-bf79-ffce65ba38be admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:17.115232 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b4693-71ec-4121-bf79-ffce65ba38be admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:17.219606 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701adb53-c078-4ef8-bcd1-fce9073b7651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:17.228603 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701adb53-c078-4ef8-bcd1-fce9073b7651 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:17.230454 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701adb53-c078-4ef8-bcd1-fce9073b7651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:17.230758 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701adb53-c078-4ef8-bcd1-fce9073b7651 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:17.231534 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701adb53-c078-4ef8-bcd1-fce9073b7651 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:17.233574 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701adb53-c078-4ef8-bcd1-fce9073b7651 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:17.261652 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1361/2733] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:29:17 2022] POST /identity/v3/users => generated 292 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:29:17.265531 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72b7f3-cbd8-4936-9d01-45ddb789ab09 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:17.276385 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72b7f3-cbd8-4936-9d01-45ddb789ab09 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:17.281862 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3245af-c39c-447a-9300-6338941f7078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:17.282529 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3245af-c39c-447a-9300-6338941f7078 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:17.282813 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3245af-c39c-447a-9300-6338941f7078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:17.283086 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3245af-c39c-447a-9300-6338941f7078 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:17.284445 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3245af-c39c-447a-9300-6338941f7078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:17.286300 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3245af-c39c-447a-9300-6338941f7078 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:17.301767 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1373/2734] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:29:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:29:17.302995 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701adb53-c078-4ef8-bcd1-fce9073b7651 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:17.314930 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701adb53-c078-4ef8-bcd1-fce9073b7651 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:17.320246 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3850be3f-1a8c-459f-bce1-cba9c8242df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:17.320917 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3850be3f-1a8c-459f-bce1-cba9c8242df7 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:17.321202 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3850be3f-1a8c-459f-bce1-cba9c8242df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:17.321484 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3850be3f-1a8c-459f-bce1-cba9c8242df7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:17.322214 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3850be3f-1a8c-459f-bce1-cba9c8242df7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:17.323851 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3850be3f-1a8c-459f-bce1-cba9c8242df7 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:17.329976 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:17.342069 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3245af-c39c-447a-9300-6338941f7078 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:17.349155 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1362/2735] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:29:17 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:29:17.353283 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3245af-c39c-447a-9300-6338941f7078 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:17.362626 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9690758-3513-4ca8-ad95-5f34e6222b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:17.363609 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9690758-3513-4ca8-ad95-5f34e6222b1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:17.363892 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9690758-3513-4ca8-ad95-5f34e6222b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:17.364365 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9690758-3513-4ca8-ad95-5f34e6222b1c admin admin] PATH_INFO: `/v3/projects/bb6b0154ed14484faa0cd05be89d773a/users/546e2607524043899a48009b3642f34a/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:17.380963 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9690758-3513-4ca8-ad95-5f34e6222b1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb6b0154ed14484faa0cd05be89d773a, user_id=546e2607524043899a48009b3642f34a, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:17.383691 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9690758-3513-4ca8-ad95-5f34e6222b1c admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:17.393960 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:17.411421 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1374/2736] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:29:17 2022] PUT /identity/v3/projects/bb6b0154ed14484faa0cd05be89d773a/users/546e2607524043899a48009b3642f34a/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:29:17.413246 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7591df-9eeb-40c9-9f8e-8607f4d71dbf None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:17.413541 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7591df-9eeb-40c9-9f8e-8607f4d71dbf None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:17.413849 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7591df-9eeb-40c9-9f8e-8607f4d71dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:17.441438 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-1f7591df-9eeb-40c9-9f8e-8607f4d71dbf None None] MFA Rules not processed for user `546e2607524043899a48009b3642f34a`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:29:17.442297 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1f7591df-9eeb-40c9-9f8e-8607f4d71dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:17.518014 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1363/2737] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:29:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:29:17.520920 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9690758-3513-4ca8-ad95-5f34e6222b1c admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:17.609444 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9690758-3513-4ca8-ad95-5f34e6222b1c admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:17.615209 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9465d76f-9be6-4790-b964-fc83120a3a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:17.615874 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9465d76f-9be6-4790-b964-fc83120a3a9e admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:17.616260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9465d76f-9be6-4790-b964-fc83120a3a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:17.616543 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9465d76f-9be6-4790-b964-fc83120a3a9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:17.617244 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9465d76f-9be6-4790-b964-fc83120a3a9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:17.619493 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9465d76f-9be6-4790-b964-fc83120a3a9e admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:17.650276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1375/2738] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:29:17 2022] POST /identity/v3/projects => generated 381 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:29:17.654337 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7591df-9eeb-40c9-9f8e-8607f4d71dbf None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:17.669849 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7591df-9eeb-40c9-9f8e-8607f4d71dbf None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:17.737173 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2ab02-920e-4458-a202-4356386d172b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:17.738004 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2ab02-920e-4458-a202-4356386d172b admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:17.739048 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2ab02-920e-4458-a202-4356386d172b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:17.739048 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2ab02-920e-4458-a202-4356386d172b admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:17.739509 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2ab02-920e-4458-a202-4356386d172b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:17.741291 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2ab02-920e-4458-a202-4356386d172b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:17.780261 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1364/2739] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:29:17 2022] POST /identity/v3/users => generated 297 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:29:17.781714 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9465d76f-9be6-4790-b964-fc83120a3a9e admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:17.792900 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9465d76f-9be6-4790-b964-fc83120a3a9e admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:17.797739 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e3298-1d8b-4124-b84e-0ebaa548c28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:17.798512 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e3298-1d8b-4124-b84e-0ebaa548c28d admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:17.798878 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e3298-1d8b-4124-b84e-0ebaa548c28d admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:17.799224 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e3298-1d8b-4124-b84e-0ebaa548c28d admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:17.800112 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25e3298-1d8b-4124-b84e-0ebaa548c28d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:17.801797 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25e3298-1d8b-4124-b84e-0ebaa548c28d admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:17.812962 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1376/2740] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:29:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:29:17.817675 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2ab02-920e-4458-a202-4356386d172b admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:17.828194 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2ab02-920e-4458-a202-4356386d172b admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:17.833577 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8409a0-12d7-40ca-b7e6-a7570cbcff4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:17.834411 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8409a0-12d7-40ca-b7e6-a7570cbcff4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:17.834696 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8409a0-12d7-40ca-b7e6-a7570cbcff4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:17.834975 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8409a0-12d7-40ca-b7e6-a7570cbcff4e admin admin] PATH_INFO: `/v3/projects/c5fc03da1de94797bcfc5475283b9e83/users/73ce6745f35d4a9e8026fe1599e96b2f/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:17.851394 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8409a0-12d7-40ca-b7e6-a7570cbcff4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5fc03da1de94797bcfc5475283b9e83, user_id=73ce6745f35d4a9e8026fe1599e96b2f, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:17.854058 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8409a0-12d7-40ca-b7e6-a7570cbcff4e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:17.867105 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1365/2741] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:29:17 2022] PUT /identity/v3/projects/c5fc03da1de94797bcfc5475283b9e83/users/73ce6745f35d4a9e8026fe1599e96b2f/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:17.871030 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e3298-1d8b-4124-b84e-0ebaa548c28d admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:17.882270 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e3298-1d8b-4124-b84e-0ebaa548c28d admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:17.942995 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63b6da-0b8f-4983-bdb1-c84e72257a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:17.943760 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63b6da-0b8f-4983-bdb1-c84e72257a17 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:17.944044 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63b6da-0b8f-4983-bdb1-c84e72257a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:17.944321 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63b6da-0b8f-4983-bdb1-c84e72257a17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:17.945056 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e63b6da-0b8f-4983-bdb1-c84e72257a17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:17.946677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e63b6da-0b8f-4983-bdb1-c84e72257a17 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:17.958993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1377/2742] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:29:17 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:29:17.962952 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8409a0-12d7-40ca-b7e6-a7570cbcff4e admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:17.973734 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8409a0-12d7-40ca-b7e6-a7570cbcff4e admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:17.978481 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d64839-5b24-48ee-8c5a-59793e1434f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:17.979411 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d64839-5b24-48ee-8c5a-59793e1434f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:17.979695 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d64839-5b24-48ee-8c5a-59793e1434f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:17.979971 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d64839-5b24-48ee-8c5a-59793e1434f7 admin admin] PATH_INFO: `/v3/projects/c5fc03da1de94797bcfc5475283b9e83/users/73ce6745f35d4a9e8026fe1599e96b2f/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:17.981964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d64839-5b24-48ee-8c5a-59793e1434f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5fc03da1de94797bcfc5475283b9e83, user_id=73ce6745f35d4a9e8026fe1599e96b2f, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:17.984667 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d64839-5b24-48ee-8c5a-59793e1434f7 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:17.996509 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1366/2743] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:29:17 2022] PUT /identity/v3/projects/c5fc03da1de94797bcfc5475283b9e83/users/73ce6745f35d4a9e8026fe1599e96b2f/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:18.008622 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62b010-8344-403b-9781-ddb6ba9371d6 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:18.008915 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62b010-8344-403b-9781-ddb6ba9371d6 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:18.009192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62b010-8344-403b-9781-ddb6ba9371d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:18.037584 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-cc62b010-8344-403b-9781-ddb6ba9371d6 None None] MFA Rules not processed for user `73ce6745f35d4a9e8026fe1599e96b2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:29:18.038839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-cc62b010-8344-403b-9781-ddb6ba9371d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:18.103652 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:18.115865 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1378/2744] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:29:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:29:18.127476 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d64839-5b24-48ee-8c5a-59793e1434f7 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:18.130733 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e6d64839-5b24-48ee-8c5a-59793e1434f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:18.179495 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:18.194278 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d64839-5b24-48ee-8c5a-59793e1434f7 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:18.198041 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037bbcb1-e2c6-448b-baa0-d920db246f7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:18.198689 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037bbcb1-e2c6-448b-baa0-d920db246f7a service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:18.198973 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037bbcb1-e2c6-448b-baa0-d920db246f7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:18.199249 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037bbcb1-e2c6-448b-baa0-d920db246f7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:18.208535 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037bbcb1-e2c6-448b-baa0-d920db246f7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:18.209961 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037bbcb1-e2c6-448b-baa0-d920db246f7a service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:18.219662 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1367/2745] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:29:18 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:29:18.329520 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc62b010-8344-403b-9781-ddb6ba9371d6 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:18.339317 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc62b010-8344-403b-9781-ddb6ba9371d6 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:18.343376 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e1c2b-a69a-45e4-82a3-f8f499ee04ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:18.344034 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e1c2b-a69a-45e4-82a3-f8f499ee04ab service cinder] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:18.344323 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e1c2b-a69a-45e4-82a3-f8f499ee04ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:18.344602 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e1c2b-a69a-45e4-82a3-f8f499ee04ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:18.407263 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1e1c2b-a69a-45e4-82a3-f8f499ee04ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:18.408753 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1e1c2b-a69a-45e4-82a3-f8f499ee04ab service cinder] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:18.418092 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1379/2746] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:29:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:29:20.937174 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037bbcb1-e2c6-448b-baa0-d920db246f7a service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:20.938857 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-037bbcb1-e2c6-448b-baa0-d920db246f7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:21.009593 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037bbcb1-e2c6-448b-baa0-d920db246f7a service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:21.015081 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa83cc51-01ba-495c-b303-a7328eb08b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:21.015991 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83cc51-01ba-495c-b303-a7328eb08b54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:21.016275 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83cc51-01ba-495c-b303-a7328eb08b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:21.016554 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83cc51-01ba-495c-b303-a7328eb08b54 admin admin] PATH_INFO: `/v3/users/331c97a12f1b4c2eb1eefa61056ce089` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:21.017734 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa83cc51-01ba-495c-b303-a7328eb08b54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=331c97a12f1b4c2eb1eefa61056ce089)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:21.020254 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa83cc51-01ba-495c-b303-a7328eb08b54 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:21.065828 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-fa83cc51-01ba-495c-b303-a7328eb08b54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '331c97a12f1b4c2eb1eefa61056ce089'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:21.072899 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-fa83cc51-01ba-495c-b303-a7328eb08b54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '331c97a12f1b4c2eb1eefa61056ce089'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:21.085716 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-fa83cc51-01ba-495c-b303-a7328eb08b54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '331c97a12f1b4c2eb1eefa61056ce089'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:21.086347 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-fa83cc51-01ba-495c-b303-a7328eb08b54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '331c97a12f1b4c2eb1eefa61056ce089'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:21.092952 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-fa83cc51-01ba-495c-b303-a7328eb08b54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '331c97a12f1b4c2eb1eefa61056ce089'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:21.099252 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1368/2747] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:29:20 2022] DELETE /identity/v3/users/331c97a12f1b4c2eb1eefa61056ce089 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:21.275536 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e1c2b-a69a-45e4-82a3-f8f499ee04ab service cinder] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:21.288020 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e1c2b-a69a-45e4-82a3-f8f499ee04ab service cinder] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:21.344604 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca448a1-74ec-4307-896a-1c5ce857365c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:21.345468 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca448a1-74ec-4307-896a-1c5ce857365c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:21.345752 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca448a1-74ec-4307-896a-1c5ce857365c admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:21.346527 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca448a1-74ec-4307-896a-1c5ce857365c admin admin] PATH_INFO: `/v3/projects/7d586c0c8dd34398af9d52425eb2bde3` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:21.347741 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca448a1-74ec-4307-896a-1c5ce857365c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d586c0c8dd34398af9d52425eb2bde3)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:21.349678 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca448a1-74ec-4307-896a-1c5ce857365c admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:21.400218 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-9ca448a1-74ec-4307-896a-1c5ce857365c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d586c0c8dd34398af9d52425eb2bde3', 'request_id': 'req-9ca448a1-74ec-4307-896a-1c5ce857365c'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:21.406500 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-9ca448a1-74ec-4307-896a-1c5ce857365c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d586c0c8dd34398af9d52425eb2bde3', 'request_id': 'req-9ca448a1-74ec-4307-896a-1c5ce857365c'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:21.413524 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-9ca448a1-74ec-4307-896a-1c5ce857365c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d586c0c8dd34398af9d52425eb2bde3', 'request_id': 'req-9ca448a1-74ec-4307-896a-1c5ce857365c'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:21.423967 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-9ca448a1-74ec-4307-896a-1c5ce857365c admin admin] The token cache is being invalidate because project 7d586c0c8dd34398af9d52425eb2bde3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:29:21.424254 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-9ca448a1-74ec-4307-896a-1c5ce857365c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:21.424866 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-9ca448a1-74ec-4307-896a-1c5ce857365c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:21.426399 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1380/2748] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:29:21 2022] DELETE /identity/v3/projects/7d586c0c8dd34398af9d52425eb2bde3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:21.430412 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa83cc51-01ba-495c-b303-a7328eb08b54 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:21.431872 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-fa83cc51-01ba-495c-b303-a7328eb08b54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:21.445328 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa83cc51-01ba-495c-b303-a7328eb08b54 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:21.500650 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34418207-f2fd-4d3a-8596-bbc45a090529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:21.501515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34418207-f2fd-4d3a-8596-bbc45a090529 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:21.501796 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34418207-f2fd-4d3a-8596-bbc45a090529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:21.502076 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34418207-f2fd-4d3a-8596-bbc45a090529 admin admin] PATH_INFO: `/v3/users/31fad66b7380445a8ba7e13c6d443df0` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:21.509716 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:21.517109 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34418207-f2fd-4d3a-8596-bbc45a090529 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31fad66b7380445a8ba7e13c6d443df0)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:21.520017 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34418207-f2fd-4d3a-8596-bbc45a090529 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:21.564383 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-34418207-f2fd-4d3a-8596-bbc45a090529 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31fad66b7380445a8ba7e13c6d443df0'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:21.571774 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-34418207-f2fd-4d3a-8596-bbc45a090529 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31fad66b7380445a8ba7e13c6d443df0'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:21.584064 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-34418207-f2fd-4d3a-8596-bbc45a090529 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31fad66b7380445a8ba7e13c6d443df0'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:21.584710 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-34418207-f2fd-4d3a-8596-bbc45a090529 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31fad66b7380445a8ba7e13c6d443df0'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:21.591495 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-34418207-f2fd-4d3a-8596-bbc45a090529 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31fad66b7380445a8ba7e13c6d443df0'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:21.597661 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1369/2749] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:29:21 2022] DELETE /identity/v3/users/31fad66b7380445a8ba7e13c6d443df0 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:21.771501 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca448a1-74ec-4307-896a-1c5ce857365c admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:21.773063 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-9ca448a1-74ec-4307-896a-1c5ce857365c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:21.787982 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca448a1-74ec-4307-896a-1c5ce857365c admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:21.809739 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:21.847192 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b15885-2e5f-43d9-b7ce-41f2902f930b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:21.848083 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b15885-2e5f-43d9-b7ce-41f2902f930b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:21.848364 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b15885-2e5f-43d9-b7ce-41f2902f930b admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:21.848638 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b15885-2e5f-43d9-b7ce-41f2902f930b admin admin] PATH_INFO: `/v3/projects/d3190959220e4ae0be2e9d8e696477ba` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:21.858815 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b15885-2e5f-43d9-b7ce-41f2902f930b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3190959220e4ae0be2e9d8e696477ba)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:21.860724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b15885-2e5f-43d9-b7ce-41f2902f930b admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:21.906777 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-b0b15885-2e5f-43d9-b7ce-41f2902f930b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3190959220e4ae0be2e9d8e696477ba', 'request_id': 'req-b0b15885-2e5f-43d9-b7ce-41f2902f930b'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:21.913260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-b0b15885-2e5f-43d9-b7ce-41f2902f930b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3190959220e4ae0be2e9d8e696477ba', 'request_id': 'req-b0b15885-2e5f-43d9-b7ce-41f2902f930b'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:21.920109 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-b0b15885-2e5f-43d9-b7ce-41f2902f930b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3190959220e4ae0be2e9d8e696477ba', 'request_id': 'req-b0b15885-2e5f-43d9-b7ce-41f2902f930b'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:21.930446 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-b0b15885-2e5f-43d9-b7ce-41f2902f930b admin admin] The token cache is being invalidate because project d3190959220e4ae0be2e9d8e696477ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:29:21.930731 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-b0b15885-2e5f-43d9-b7ce-41f2902f930b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:21.931398 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-b0b15885-2e5f-43d9-b7ce-41f2902f930b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:21.932936 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1381/2750] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:29:21 2022] DELETE /identity/v3/projects/d3190959220e4ae0be2e9d8e696477ba => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:24.171043 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34418207-f2fd-4d3a-8596-bbc45a090529 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:24.172535 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-34418207-f2fd-4d3a-8596-bbc45a090529 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:24.185519 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34418207-f2fd-4d3a-8596-bbc45a090529 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:24.248780 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac48f8-4f26-4a8d-94af-bfb26c2152e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:24.249668 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac48f8-4f26-4a8d-94af-bfb26c2152e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:24.249953 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac48f8-4f26-4a8d-94af-bfb26c2152e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:24.250234 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac48f8-4f26-4a8d-94af-bfb26c2152e0 admin admin] PATH_INFO: `/v3/users/546e2607524043899a48009b3642f34a` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:24.251476 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aac48f8-4f26-4a8d-94af-bfb26c2152e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=546e2607524043899a48009b3642f34a)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:24.253546 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aac48f8-4f26-4a8d-94af-bfb26c2152e0 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:24.307202 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3aac48f8-4f26-4a8d-94af-bfb26c2152e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '546e2607524043899a48009b3642f34a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:24.314315 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3aac48f8-4f26-4a8d-94af-bfb26c2152e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '546e2607524043899a48009b3642f34a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:24.326577 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3aac48f8-4f26-4a8d-94af-bfb26c2152e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '546e2607524043899a48009b3642f34a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:24.327231 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3aac48f8-4f26-4a8d-94af-bfb26c2152e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '546e2607524043899a48009b3642f34a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:24.329359 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:24.335762 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3aac48f8-4f26-4a8d-94af-bfb26c2152e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '546e2607524043899a48009b3642f34a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:24.342477 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1370/2751] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:29:24 2022] DELETE /identity/v3/users/546e2607524043899a48009b3642f34a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:24.347145 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b433c1f-a7f1-461b-a146-8690c1d11c97 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:24.347668 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b433c1f-a7f1-461b-a146-8690c1d11c97 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:24.347782 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b433c1f-a7f1-461b-a146-8690c1d11c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:24.364808 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-8b433c1f-a7f1-461b-a146-8690c1d11c97 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:29:24.366002 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-8b433c1f-a7f1-461b-a146-8690c1d11c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:24.427010 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1382/2752] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:29:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:29:24.437227 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac48f8-4f26-4a8d-94af-bfb26c2152e0 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:24.439228 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3aac48f8-4f26-4a8d-94af-bfb26c2152e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:24.499562 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac48f8-4f26-4a8d-94af-bfb26c2152e0 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:24.503796 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49363d0b-61e4-44b0-83f6-8c3c14bf2704 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:24.504444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49363d0b-61e4-44b0-83f6-8c3c14bf2704 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:24.504732 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49363d0b-61e4-44b0-83f6-8c3c14bf2704 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:24.505013 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49363d0b-61e4-44b0-83f6-8c3c14bf2704 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:24.513876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49363d0b-61e4-44b0-83f6-8c3c14bf2704 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:24.515339 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49363d0b-61e4-44b0-83f6-8c3c14bf2704 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:24.525253 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1371/2753] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:29:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:29:24.785064 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b433c1f-a7f1-461b-a146-8690c1d11c97 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:24.786511 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-8b433c1f-a7f1-461b-a146-8690c1d11c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:24.801199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b433c1f-a7f1-461b-a146-8690c1d11c97 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:24.807062 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da68353b-783a-48eb-8cc2-9885866f1366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:24.807990 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da68353b-783a-48eb-8cc2-9885866f1366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:24.808289 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da68353b-783a-48eb-8cc2-9885866f1366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:24.808545 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da68353b-783a-48eb-8cc2-9885866f1366 admin admin] PATH_INFO: `/v3/projects/bb6b0154ed14484faa0cd05be89d773a` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:24.809813 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da68353b-783a-48eb-8cc2-9885866f1366 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb6b0154ed14484faa0cd05be89d773a)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:24.811669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da68353b-783a-48eb-8cc2-9885866f1366 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:24.853123 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:24.860243 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-da68353b-783a-48eb-8cc2-9885866f1366 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb6b0154ed14484faa0cd05be89d773a', 'request_id': 'req-da68353b-783a-48eb-8cc2-9885866f1366'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:24.867435 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-da68353b-783a-48eb-8cc2-9885866f1366 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb6b0154ed14484faa0cd05be89d773a', 'request_id': 'req-da68353b-783a-48eb-8cc2-9885866f1366'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:24.876666 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-da68353b-783a-48eb-8cc2-9885866f1366 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb6b0154ed14484faa0cd05be89d773a', 'request_id': 'req-da68353b-783a-48eb-8cc2-9885866f1366'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:24.887220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-da68353b-783a-48eb-8cc2-9885866f1366 admin admin] The token cache is being invalidate because project bb6b0154ed14484faa0cd05be89d773a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:29:24.887543 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-da68353b-783a-48eb-8cc2-9885866f1366 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:24.888246 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-da68353b-783a-48eb-8cc2-9885866f1366 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:24.889957 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1383/2754] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:29:24 2022] DELETE /identity/v3/projects/bb6b0154ed14484faa0cd05be89d773a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:24.894604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49363d0b-61e4-44b0-83f6-8c3c14bf2704 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:24.905802 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49363d0b-61e4-44b0-83f6-8c3c14bf2704 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:24.961734 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8226c87a-ea0d-4d4b-82e5-dff0275ba576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:24.962610 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226c87a-ea0d-4d4b-82e5-dff0275ba576 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:24.962900 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226c87a-ea0d-4d4b-82e5-dff0275ba576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:24.963180 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226c87a-ea0d-4d4b-82e5-dff0275ba576 admin admin] PATH_INFO: `/v3/users/73ce6745f35d4a9e8026fe1599e96b2f` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:24.964408 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8226c87a-ea0d-4d4b-82e5-dff0275ba576 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73ce6745f35d4a9e8026fe1599e96b2f)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:24.966341 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8226c87a-ea0d-4d4b-82e5-dff0275ba576 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:25.011674 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-8226c87a-ea0d-4d4b-82e5-dff0275ba576 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73ce6745f35d4a9e8026fe1599e96b2f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:25.019562 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-8226c87a-ea0d-4d4b-82e5-dff0275ba576 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73ce6745f35d4a9e8026fe1599e96b2f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:25.031961 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-8226c87a-ea0d-4d4b-82e5-dff0275ba576 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73ce6745f35d4a9e8026fe1599e96b2f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:25.032621 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-8226c87a-ea0d-4d4b-82e5-dff0275ba576 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73ce6745f35d4a9e8026fe1599e96b2f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:25.039988 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-8226c87a-ea0d-4d4b-82e5-dff0275ba576 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73ce6745f35d4a9e8026fe1599e96b2f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:25.045789 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1372/2755] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:29:24 2022] DELETE /identity/v3/users/73ce6745f35d4a9e8026fe1599e96b2f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:25.315148 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da68353b-783a-48eb-8cc2-9885866f1366 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:25.320137 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-da68353b-783a-48eb-8cc2-9885866f1366 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:25.346772 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da68353b-783a-48eb-8cc2-9885866f1366 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:25.413048 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28910b25-86a5-43b2-8eac-4b6526f51b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:25.413920 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28910b25-86a5-43b2-8eac-4b6526f51b11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:25.414205 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28910b25-86a5-43b2-8eac-4b6526f51b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:25.414485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28910b25-86a5-43b2-8eac-4b6526f51b11 admin admin] PATH_INFO: `/v3/projects/c5fc03da1de94797bcfc5475283b9e83` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:25.415709 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28910b25-86a5-43b2-8eac-4b6526f51b11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5fc03da1de94797bcfc5475283b9e83)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:25.417647 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28910b25-86a5-43b2-8eac-4b6526f51b11 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:25.465929 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-28910b25-86a5-43b2-8eac-4b6526f51b11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5fc03da1de94797bcfc5475283b9e83', 'request_id': 'req-28910b25-86a5-43b2-8eac-4b6526f51b11'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:25.472323 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-28910b25-86a5-43b2-8eac-4b6526f51b11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5fc03da1de94797bcfc5475283b9e83', 'request_id': 'req-28910b25-86a5-43b2-8eac-4b6526f51b11'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:25.483733 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-28910b25-86a5-43b2-8eac-4b6526f51b11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5fc03da1de94797bcfc5475283b9e83', 'request_id': 'req-28910b25-86a5-43b2-8eac-4b6526f51b11'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:25.495316 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-28910b25-86a5-43b2-8eac-4b6526f51b11 admin admin] The token cache is being invalidate because project c5fc03da1de94797bcfc5475283b9e83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:29:25.495618 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-28910b25-86a5-43b2-8eac-4b6526f51b11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:25.496878 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-28910b25-86a5-43b2-8eac-4b6526f51b11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:25.498928 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1384/2756] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:29:25 2022] DELETE /identity/v3/projects/c5fc03da1de94797bcfc5475283b9e83 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:25.504243 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e793f85-0a66-42f6-88b6-de5e159fafa5 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:25.504707 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e793f85-0a66-42f6-88b6-de5e159fafa5 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:25.504927 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e793f85-0a66-42f6-88b6-de5e159fafa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:25.522648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-3e793f85-0a66-42f6-88b6-de5e159fafa5 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:29:25.523996 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3e793f85-0a66-42f6-88b6-de5e159fafa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:25.532669 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:25.588087 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1373/2757] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:29:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:29:25.596117 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a2dac-17d8-4320-8bf5-7b3de8a4907d None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:25.596337 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a2dac-17d8-4320-8bf5-7b3de8a4907d None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:25.596653 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a2dac-17d8-4320-8bf5-7b3de8a4907d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:25.615886 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-6d5a2dac-17d8-4320-8bf5-7b3de8a4907d None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:29:25.617116 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-6d5a2dac-17d8-4320-8bf5-7b3de8a4907d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:25.625349 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1385/2758] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:29:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:29:25.629952 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e793f85-0a66-42f6-88b6-de5e159fafa5 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:25.640978 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e793f85-0a66-42f6-88b6-de5e159fafa5 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:25.646896 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ca004-27e6-4d47-bd85-ad764295b67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:25.647724 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ca004-27e6-4d47-bd85-ad764295b67b admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:25.647999 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ca004-27e6-4d47-bd85-ad764295b67b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:25.648312 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ca004-27e6-4d47-bd85-ad764295b67b admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:25.649056 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ca004-27e6-4d47-bd85-ad764295b67b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:29:25.649368 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ca004-27e6-4d47-bd85-ad764295b67b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:25.650986 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ca004-27e6-4d47-bd85-ad764295b67b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:25.667060 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1374/2759] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:29:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:29:25.667780 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ca004-27e6-4d47-bd85-ad764295b67b admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:25.667902 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a2dac-17d8-4320-8bf5-7b3de8a4907d None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:25.669452 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7c8ca004-27e6-4d47-bd85-ad764295b67b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:25.677606 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a2dac-17d8-4320-8bf5-7b3de8a4907d None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:25.682323 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74814556-6399-48dc-8ed9-a6fa17102a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:25.682571 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ca004-27e6-4d47-bd85-ad764295b67b admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:25.682975 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74814556-6399-48dc-8ed9-a6fa17102a83 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:25.683264 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74814556-6399-48dc-8ed9-a6fa17102a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:25.683586 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74814556-6399-48dc-8ed9-a6fa17102a83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:25.684287 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74814556-6399-48dc-8ed9-a6fa17102a83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:25.686181 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74814556-6399-48dc-8ed9-a6fa17102a83 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:25.687122 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12571b9b-6233-4598-b1d6-31cdfcc0a4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:25.688031 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12571b9b-6233-4598-b1d6-31cdfcc0a4e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:25.688285 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12571b9b-6233-4598-b1d6-31cdfcc0a4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:25.688562 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12571b9b-6233-4598-b1d6-31cdfcc0a4e9 admin admin] PATH_INFO: `/v3/users/b81617c1e30240649db32ec6e2b11227` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:25.689736 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12571b9b-6233-4598-b1d6-31cdfcc0a4e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b81617c1e30240649db32ec6e2b11227)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:25.692513 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12571b9b-6233-4598-b1d6-31cdfcc0a4e9 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:25.715773 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1386/2760] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:29:25 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:29:25.716902 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74814556-6399-48dc-8ed9-a6fa17102a83 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:25.742646 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-12571b9b-6233-4598-b1d6-31cdfcc0a4e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b81617c1e30240649db32ec6e2b11227'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:25.750575 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-12571b9b-6233-4598-b1d6-31cdfcc0a4e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b81617c1e30240649db32ec6e2b11227'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:25.751082 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:25.764825 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-12571b9b-6233-4598-b1d6-31cdfcc0a4e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b81617c1e30240649db32ec6e2b11227'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:25.765527 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-12571b9b-6233-4598-b1d6-31cdfcc0a4e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b81617c1e30240649db32ec6e2b11227'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:25.772423 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-12571b9b-6233-4598-b1d6-31cdfcc0a4e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b81617c1e30240649db32ec6e2b11227'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:25.778669 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1375/2761] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:29:25 2022] DELETE /identity/v3/users/b81617c1e30240649db32ec6e2b11227 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:25.782964 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d645d9-43f8-42a6-b4eb-7c4a82562f08 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:25.783118 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74814556-6399-48dc-8ed9-a6fa17102a83 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:25.783278 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d645d9-43f8-42a6-b4eb-7c4a82562f08 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:25.783553 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d645d9-43f8-42a6-b4eb-7c4a82562f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:25.800601 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-d2d645d9-43f8-42a6-b4eb-7c4a82562f08 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:29:25.801718 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d2d645d9-43f8-42a6-b4eb-7c4a82562f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:25.838789 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be7f4a0-f270-454b-9cf8-900ff70d5a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:25.839495 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be7f4a0-f270-454b-9cf8-900ff70d5a62 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:25.839781 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be7f4a0-f270-454b-9cf8-900ff70d5a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:25.840059 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be7f4a0-f270-454b-9cf8-900ff70d5a62 admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:25.840747 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be7f4a0-f270-454b-9cf8-900ff70d5a62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:25.842645 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be7f4a0-f270-454b-9cf8-900ff70d5a62 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:25.861398 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1376/2762] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:29:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:29:25.867398 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1387/2763] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:29:25 2022] POST /identity/v3/users => generated 283 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:29:25.868566 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d645d9-43f8-42a6-b4eb-7c4a82562f08 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:25.869937 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d2d645d9-43f8-42a6-b4eb-7c4a82562f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:25.871770 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be7f4a0-f270-454b-9cf8-900ff70d5a62 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:25.873354 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-1be7f4a0-f270-454b-9cf8-900ff70d5a62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:25.881773 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d645d9-43f8-42a6-b4eb-7c4a82562f08 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:25.886412 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258af761-b08e-4aba-887d-53c089e6be6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:25.887072 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258af761-b08e-4aba-887d-53c089e6be6b admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:25.887393 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258af761-b08e-4aba-887d-53c089e6be6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:25.887678 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258af761-b08e-4aba-887d-53c089e6be6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:25.888534 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258af761-b08e-4aba-887d-53c089e6be6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:25.890136 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258af761-b08e-4aba-887d-53c089e6be6b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:25.900374 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1377/2764] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:29:25.904328 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258af761-b08e-4aba-887d-53c089e6be6b admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:25.909762 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:25.915062 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258af761-b08e-4aba-887d-53c089e6be6b admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:25.922213 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06189b24-815b-4362-b265-a0535153c3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:25.923172 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06189b24-815b-4362-b265-a0535153c3b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:25.923725 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06189b24-815b-4362-b265-a0535153c3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:25.924178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06189b24-815b-4362-b265-a0535153c3b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:25.925745 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06189b24-815b-4362-b265-a0535153c3b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:25.927375 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06189b24-815b-4362-b265-a0535153c3b8 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:25.934240 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be7f4a0-f270-454b-9cf8-900ff70d5a62 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:25.937604 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1378/2765] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:29:25.937950 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945e802-35d4-4560-b54c-5ca404db42d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:25.938668 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945e802-35d4-4560-b54c-5ca404db42d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:25.938959 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945e802-35d4-4560-b54c-5ca404db42d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:25.939241 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945e802-35d4-4560-b54c-5ca404db42d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:25.941705 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06189b24-815b-4362-b265-a0535153c3b8 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:25.948674 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6945e802-35d4-4560-b54c-5ca404db42d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:25.950139 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6945e802-35d4-4560-b54c-5ca404db42d7 service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:25.951798 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06189b24-815b-4362-b265-a0535153c3b8 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:25.957021 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040bbcf-2d58-4f8c-8f36-698c94175549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:25.960104 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1388/2766] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:29:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 16:29:25.962187 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040bbcf-2d58-4f8c-8f36-698c94175549 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:25.962480 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040bbcf-2d58-4f8c-8f36-698c94175549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:25.962808 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040bbcf-2d58-4f8c-8f36-698c94175549 admin admin] PATH_INFO: `/v3/projects/45026d08db0c4a82a177078744d32801/users/3853921a5a414013be902cfd984b9248/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:25.978768 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4040bbcf-2d58-4f8c-8f36-698c94175549 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45026d08db0c4a82a177078744d32801, user_id=3853921a5a414013be902cfd984b9248, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:25.981553 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4040bbcf-2d58-4f8c-8f36-698c94175549 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:25.999419 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1379/2767] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:29:25 2022] PUT /identity/v3/projects/45026d08db0c4a82a177078744d32801/users/3853921a5a414013be902cfd984b9248/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:29:26.008309 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff979c8a-eae2-4ac0-b790-4b823460c381 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:26.008599 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff979c8a-eae2-4ac0-b790-4b823460c381 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:26.008974 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff979c8a-eae2-4ac0-b790-4b823460c381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:26.039608 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-ff979c8a-eae2-4ac0-b790-4b823460c381 None None] MFA Rules not processed for user `3853921a5a414013be902cfd984b9248`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:29:26.039608 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-ff979c8a-eae2-4ac0-b790-4b823460c381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:26.109424 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:26.122842 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1389/2768] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:29:26 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:29:26.123814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040bbcf-2d58-4f8c-8f36-698c94175549 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:26.192045 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040bbcf-2d58-4f8c-8f36-698c94175549 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:26.196897 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d753c74-a44b-4445-ab60-ea36cef78159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:26.197563 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d753c74-a44b-4445-ab60-ea36cef78159 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:26.197850 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d753c74-a44b-4445-ab60-ea36cef78159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:26.198129 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d753c74-a44b-4445-ab60-ea36cef78159 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:26.198824 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d753c74-a44b-4445-ab60-ea36cef78159 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:26.200796 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d753c74-a44b-4445-ab60-ea36cef78159 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:26.228380 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1380/2769] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:29:26 2022] POST /identity/v3/projects => generated 365 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:29:26.232531 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff979c8a-eae2-4ac0-b790-4b823460c381 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:26.245164 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff979c8a-eae2-4ac0-b790-4b823460c381 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:26.264169 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d753c74-a44b-4445-ab60-ea36cef78159 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:26.266355 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3d753c74-a44b-4445-ab60-ea36cef78159 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:26.312477 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251edc0-8030-4c69-9a25-527525f6d7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:26.313085 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251edc0-8030-4c69-9a25-527525f6d7a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:26.313374 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251edc0-8030-4c69-9a25-527525f6d7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:26.313662 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251edc0-8030-4c69-9a25-527525f6d7a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:26.314360 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9251edc0-8030-4c69-9a25-527525f6d7a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:26.316351 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9251edc0-8030-4c69-9a25-527525f6d7a7 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:26.337316 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1390/2770] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:29:26 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:29:26.338939 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d753c74-a44b-4445-ab60-ea36cef78159 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:26.341153 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251edc0-8030-4c69-9a25-527525f6d7a7 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:26.343993 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e92bec-105f-40bf-a3b2-f6a428372c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:26.344884 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e92bec-105f-40bf-a3b2-f6a428372c4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:26.345169 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e92bec-105f-40bf-a3b2-f6a428372c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:26.345444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e92bec-105f-40bf-a3b2-f6a428372c4f admin admin] PATH_INFO: `/v3/projects/c492f0fdd7d54d0da0b68ffd73132596` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:26.347203 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e92bec-105f-40bf-a3b2-f6a428372c4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c492f0fdd7d54d0da0b68ffd73132596)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:26.349236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e92bec-105f-40bf-a3b2-f6a428372c4f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:26.353396 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251edc0-8030-4c69-9a25-527525f6d7a7 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:26.361014 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c4316b-cdc6-487c-ad2b-e4e0b47faccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:26.361787 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4316b-cdc6-487c-ad2b-e4e0b47faccf admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:26.362126 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4316b-cdc6-487c-ad2b-e4e0b47faccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:26.362412 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4316b-cdc6-487c-ad2b-e4e0b47faccf admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:26.363141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c4316b-cdc6-487c-ad2b-e4e0b47faccf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:26.364762 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c4316b-cdc6-487c-ad2b-e4e0b47faccf admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:26.374900 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1391/2771] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:29:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:29:26.379116 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c4316b-cdc6-487c-ad2b-e4e0b47faccf admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:26.389091 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c4316b-cdc6-487c-ad2b-e4e0b47faccf admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:26.402782 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c7e92bec-105f-40bf-a3b2-f6a428372c4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c492f0fdd7d54d0da0b68ffd73132596', 'request_id': 'req-c7e92bec-105f-40bf-a3b2-f6a428372c4f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:26.409714 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c7e92bec-105f-40bf-a3b2-f6a428372c4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c492f0fdd7d54d0da0b68ffd73132596', 'request_id': 'req-c7e92bec-105f-40bf-a3b2-f6a428372c4f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:26.411383 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:26.418673 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c7e92bec-105f-40bf-a3b2-f6a428372c4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c492f0fdd7d54d0da0b68ffd73132596', 'request_id': 'req-c7e92bec-105f-40bf-a3b2-f6a428372c4f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:26.431757 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c7e92bec-105f-40bf-a3b2-f6a428372c4f admin admin] The token cache is being invalidate because project c492f0fdd7d54d0da0b68ffd73132596 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:29:26.432049 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c7e92bec-105f-40bf-a3b2-f6a428372c4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:26.433128 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c7e92bec-105f-40bf-a3b2-f6a428372c4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:26.435055 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1381/2772] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:29:26 2022] DELETE /identity/v3/projects/c492f0fdd7d54d0da0b68ffd73132596 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:26.439387 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e92bec-105f-40bf-a3b2-f6a428372c4f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:26.440854 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c7e92bec-105f-40bf-a3b2-f6a428372c4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:26.450683 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d46aa-a591-4696-924b-a9d58bb6bdd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:26.451588 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d46aa-a591-4696-924b-a9d58bb6bdd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:26.451876 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d46aa-a591-4696-924b-a9d58bb6bdd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:26.452162 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d46aa-a591-4696-924b-a9d58bb6bdd1 admin admin] PATH_INFO: `/v3/projects/a2eb893f6ff9410ab3d72def028f0495/users/0fcb7cf4b48a41fa92e924c9d80e93b5/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:26.467160 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3d46aa-a591-4696-924b-a9d58bb6bdd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2eb893f6ff9410ab3d72def028f0495, user_id=0fcb7cf4b48a41fa92e924c9d80e93b5, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:26.469918 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3d46aa-a591-4696-924b-a9d58bb6bdd1 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:26.483434 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1392/2773] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:29:26 2022] PUT /identity/v3/projects/a2eb893f6ff9410ab3d72def028f0495/users/0fcb7cf4b48a41fa92e924c9d80e93b5/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:29:26.487286 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d46aa-a591-4696-924b-a9d58bb6bdd1 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:26.488978 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-1d3d46aa-a591-4696-924b-a9d58bb6bdd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:26.505560 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e92bec-105f-40bf-a3b2-f6a428372c4f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:26.553430 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d46aa-a591-4696-924b-a9d58bb6bdd1 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:26.558199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea13858-9b5f-417e-a596-dcfa6b33b169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:26.558874 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea13858-9b5f-417e-a596-dcfa6b33b169 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:26.559160 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea13858-9b5f-417e-a596-dcfa6b33b169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:26.559473 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea13858-9b5f-417e-a596-dcfa6b33b169 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:26.560204 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea13858-9b5f-417e-a596-dcfa6b33b169 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:26.561751 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7cb583-6f5e-406d-a12e-95f569f4fce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:26.561895 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea13858-9b5f-417e-a596-dcfa6b33b169 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:26.562801 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7cb583-6f5e-406d-a12e-95f569f4fce4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:26.563032 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7cb583-6f5e-406d-a12e-95f569f4fce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:26.563351 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7cb583-6f5e-406d-a12e-95f569f4fce4 admin admin] PATH_INFO: `/v3/users/a5a402682108401cade304441a87393a` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:26.564085 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:26.564562 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7cb583-6f5e-406d-a12e-95f569f4fce4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5a402682108401cade304441a87393a)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:26.567018 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7cb583-6f5e-406d-a12e-95f569f4fce4 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:26.578360 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1393/2774] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:29:26 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:29:26.579401 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea13858-9b5f-417e-a596-dcfa6b33b169 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:26.589526 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea13858-9b5f-417e-a596-dcfa6b33b169 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:26.594481 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e5d4fc-1e17-4db3-952c-d2b60a5c51f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:26.595527 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5d4fc-1e17-4db3-952c-d2b60a5c51f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:26.595896 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5d4fc-1e17-4db3-952c-d2b60a5c51f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:26.596180 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5d4fc-1e17-4db3-952c-d2b60a5c51f2 admin admin] PATH_INFO: `/v3/projects/a2eb893f6ff9410ab3d72def028f0495/users/0fcb7cf4b48a41fa92e924c9d80e93b5/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:26.598293 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e5d4fc-1e17-4db3-952c-d2b60a5c51f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2eb893f6ff9410ab3d72def028f0495, user_id=0fcb7cf4b48a41fa92e924c9d80e93b5, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:26.601039 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e5d4fc-1e17-4db3-952c-d2b60a5c51f2 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:26.621416 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1394/2775] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:29:26 2022] PUT /identity/v3/projects/a2eb893f6ff9410ab3d72def028f0495/users/0fcb7cf4b48a41fa92e924c9d80e93b5/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:26.621657 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ca7cb583-6f5e-406d-a12e-95f569f4fce4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5a402682108401cade304441a87393a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:26.623475 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ca7cb583-6f5e-406d-a12e-95f569f4fce4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5a402682108401cade304441a87393a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:26.638371 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ca7cb583-6f5e-406d-a12e-95f569f4fce4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5a402682108401cade304441a87393a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:26.639190 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ca7cb583-6f5e-406d-a12e-95f569f4fce4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5a402682108401cade304441a87393a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:26.647850 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ca7cb583-6f5e-406d-a12e-95f569f4fce4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5a402682108401cade304441a87393a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:26.654792 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1382/2776] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:29:26 2022] DELETE /identity/v3/users/a5a402682108401cade304441a87393a => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:29:26.678949 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a08132-acfd-41e1-b23f-ad4419066666 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:26.679237 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a08132-acfd-41e1-b23f-ad4419066666 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:26.679565 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a08132-acfd-41e1-b23f-ad4419066666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:26.708063 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-93a08132-acfd-41e1-b23f-ad4419066666 None None] MFA Rules not processed for user `0fcb7cf4b48a41fa92e924c9d80e93b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:29:26.709366 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-93a08132-acfd-41e1-b23f-ad4419066666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:26.784311 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1395/2777] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:29:26 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:29:26.795758 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7cb583-6f5e-406d-a12e-95f569f4fce4 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:26.797258 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ca7cb583-6f5e-406d-a12e-95f569f4fce4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:26.811621 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:26.863268 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7cb583-6f5e-406d-a12e-95f569f4fce4 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:26.866961 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9c3eb9-11b4-4189-9f4c-e8366f618da6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:26.867637 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c3eb9-11b4-4189-9f4c-e8366f618da6 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:26.867921 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c3eb9-11b4-4189-9f4c-e8366f618da6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:26.868198 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c3eb9-11b4-4189-9f4c-e8366f618da6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:26.877308 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9c3eb9-11b4-4189-9f4c-e8366f618da6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:26.879276 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9c3eb9-11b4-4189-9f4c-e8366f618da6 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:26.889168 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1383/2778] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:29:26 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:29:26.932263 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a08132-acfd-41e1-b23f-ad4419066666 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:26.933737 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-93a08132-acfd-41e1-b23f-ad4419066666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:26.973074 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9c3eb9-11b4-4189-9f4c-e8366f618da6 service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:26.982391 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9c3eb9-11b4-4189-9f4c-e8366f618da6 service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:26.986087 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c7ef4-649f-4b5c-a03b-91bb3f73b110 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:26.986730 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c7ef4-649f-4b5c-a03b-91bb3f73b110 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:26.987021 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c7ef4-649f-4b5c-a03b-91bb3f73b110 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:26.987863 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c7ef4-649f-4b5c-a03b-91bb3f73b110 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:26.989986 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0f6c7ef4-649f-4b5c-a03b-91bb3f73b110 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:27.012566 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a08132-acfd-41e1-b23f-ad4419066666 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:27.017220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff1492-9fd2-4291-b3df-d08392e944ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:27.018108 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff1492-9fd2-4291-b3df-d08392e944ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:27.018737 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff1492-9fd2-4291-b3df-d08392e944ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:27.018737 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff1492-9fd2-4291-b3df-d08392e944ad admin admin] PATH_INFO: `/v3/projects/585ef734e97d48a69e15c06b30772988` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:27.019885 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eff1492-9fd2-4291-b3df-d08392e944ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=585ef734e97d48a69e15c06b30772988)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:27.021790 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eff1492-9fd2-4291-b3df-d08392e944ad admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:27.051806 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6c7ef4-649f-4b5c-a03b-91bb3f73b110 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:27.053297 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6c7ef4-649f-4b5c-a03b-91bb3f73b110 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:27.064547 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1384/2779] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:29:26 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:29:27.071094 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-4eff1492-9fd2-4291-b3df-d08392e944ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '585ef734e97d48a69e15c06b30772988', 'request_id': 'req-4eff1492-9fd2-4291-b3df-d08392e944ad'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:27.077443 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-4eff1492-9fd2-4291-b3df-d08392e944ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '585ef734e97d48a69e15c06b30772988', 'request_id': 'req-4eff1492-9fd2-4291-b3df-d08392e944ad'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:27.085261 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-4eff1492-9fd2-4291-b3df-d08392e944ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '585ef734e97d48a69e15c06b30772988', 'request_id': 'req-4eff1492-9fd2-4291-b3df-d08392e944ad'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:27.092760 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:27.097232 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-4eff1492-9fd2-4291-b3df-d08392e944ad admin admin] The token cache is being invalidate because project 585ef734e97d48a69e15c06b30772988 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:29:27.097820 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-4eff1492-9fd2-4291-b3df-d08392e944ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:27.099019 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-4eff1492-9fd2-4291-b3df-d08392e944ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:27.105954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1396/2780] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:29:26 2022] DELETE /identity/v3/projects/585ef734e97d48a69e15c06b30772988 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:27.107733 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db0631-5390-4735-842d-58a02198f266 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:27.108026 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db0631-5390-4735-842d-58a02198f266 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:27.108335 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db0631-5390-4735-842d-58a02198f266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:27.129072 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-96db0631-5390-4735-842d-58a02198f266 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:29:27.130597 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-96db0631-5390-4735-842d-58a02198f266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:27.194070 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1385/2781] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:29:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:29:27.203936 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8afa334-587d-4c49-82f8-1a9a664d9284 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:27.204229 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8afa334-587d-4c49-82f8-1a9a664d9284 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:27.204509 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8afa334-587d-4c49-82f8-1a9a664d9284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:27.224558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-b8afa334-587d-4c49-82f8-1a9a664d9284 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:29:27.225792 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-b8afa334-587d-4c49-82f8-1a9a664d9284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:27.233900 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1397/2782] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:29:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:29:27.238652 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db0631-5390-4735-842d-58a02198f266 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:27.249276 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db0631-5390-4735-842d-58a02198f266 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:27.255065 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a7666-98c6-4345-87a4-316c1dacd086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:27.255910 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a7666-98c6-4345-87a4-316c1dacd086 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:27.256262 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a7666-98c6-4345-87a4-316c1dacd086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:27.256612 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a7666-98c6-4345-87a4-316c1dacd086 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:27.257429 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a7666-98c6-4345-87a4-316c1dacd086 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:29:27.257808 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a7666-98c6-4345-87a4-316c1dacd086 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:27.259574 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a7666-98c6-4345-87a4-316c1dacd086 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:27.271786 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1386/2783] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:29:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:29:27.276438 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8afa334-587d-4c49-82f8-1a9a664d9284 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:27.286161 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8afa334-587d-4c49-82f8-1a9a664d9284 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:27.291590 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f29bbb-913a-449e-b009-a8dd388a04e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:27.292351 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f29bbb-913a-449e-b009-a8dd388a04e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:27.292638 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f29bbb-913a-449e-b009-a8dd388a04e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:27.292964 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f29bbb-913a-449e-b009-a8dd388a04e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:27.293693 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f29bbb-913a-449e-b009-a8dd388a04e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:27.295598 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f29bbb-913a-449e-b009-a8dd388a04e7 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:27.329200 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1398/2784] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:29:27 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:29:27.330320 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a7666-98c6-4345-87a4-316c1dacd086 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:27.340483 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a7666-98c6-4345-87a4-316c1dacd086 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:27.383017 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:27.406967 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c821acf-9f2b-4ff7-8a24-ba68719dd1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:27.407667 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c821acf-9f2b-4ff7-8a24-ba68719dd1cc admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:27.407946 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c821acf-9f2b-4ff7-8a24-ba68719dd1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:27.408224 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c821acf-9f2b-4ff7-8a24-ba68719dd1cc admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:27.408914 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c821acf-9f2b-4ff7-8a24-ba68719dd1cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:27.411326 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c821acf-9f2b-4ff7-8a24-ba68719dd1cc admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:27.431528 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1387/2785] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:29:27 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:29:27.435623 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f29bbb-913a-449e-b009-a8dd388a04e7 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:27.446104 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f29bbb-913a-449e-b009-a8dd388a04e7 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:27.450763 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474fed3-4160-42da-81d7-6baad89d3ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:27.451486 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474fed3-4160-42da-81d7-6baad89d3ad2 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:27.451772 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474fed3-4160-42da-81d7-6baad89d3ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:27.452051 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474fed3-4160-42da-81d7-6baad89d3ad2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:27.452773 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3474fed3-4160-42da-81d7-6baad89d3ad2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:27.454383 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3474fed3-4160-42da-81d7-6baad89d3ad2 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:27.464774 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1399/2786] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:29:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:29:27.468729 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c821acf-9f2b-4ff7-8a24-ba68719dd1cc admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:27.479076 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c821acf-9f2b-4ff7-8a24-ba68719dd1cc admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:27.483806 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6acee98-b246-4e81-b8db-be95f0541a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:27.484456 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6acee98-b246-4e81-b8db-be95f0541a34 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:27.484738 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6acee98-b246-4e81-b8db-be95f0541a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:27.485044 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6acee98-b246-4e81-b8db-be95f0541a34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:27.485768 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6acee98-b246-4e81-b8db-be95f0541a34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:27.487963 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6acee98-b246-4e81-b8db-be95f0541a34 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:27.497955 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1388/2787] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:29:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:29:27.502036 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474fed3-4160-42da-81d7-6baad89d3ad2 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:27.511869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474fed3-4160-42da-81d7-6baad89d3ad2 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:27.516436 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd2357-40d9-4b41-810b-f908763df4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:27.517335 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd2357-40d9-4b41-810b-f908763df4b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:27.517619 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd2357-40d9-4b41-810b-f908763df4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:27.517894 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd2357-40d9-4b41-810b-f908763df4b5 admin admin] PATH_INFO: `/v3/projects/a1f1969c4ac947e6aec774ee02c283dd/users/fd9f6984e1764a1abe892540f8d0459b/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:27.532293 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdd2357-40d9-4b41-810b-f908763df4b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1f1969c4ac947e6aec774ee02c283dd, user_id=fd9f6984e1764a1abe892540f8d0459b, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:27.535492 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdd2357-40d9-4b41-810b-f908763df4b5 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:27.548334 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1400/2788] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:29:27 2022] PUT /identity/v3/projects/a1f1969c4ac947e6aec774ee02c283dd/users/fd9f6984e1764a1abe892540f8d0459b/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:27.558113 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc3b04-7ab9-4fce-be49-2db6351bc020 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:27.558547 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc3b04-7ab9-4fce-be49-2db6351bc020 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:27.558655 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc3b04-7ab9-4fce-be49-2db6351bc020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:27.584157 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-cefc3b04-7ab9-4fce-be49-2db6351bc020 None None] MFA Rules not processed for user `fd9f6984e1764a1abe892540f8d0459b`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:29:27.585399 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-cefc3b04-7ab9-4fce-be49-2db6351bc020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:27.647421 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1389/2789] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:29:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:29:27.651750 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd2357-40d9-4b41-810b-f908763df4b5 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:27.712413 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd2357-40d9-4b41-810b-f908763df4b5 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:27.717072 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa72532-6f6c-4463-a120-294b34e6bf24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:27.717721 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa72532-6f6c-4463-a120-294b34e6bf24 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:27.718006 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa72532-6f6c-4463-a120-294b34e6bf24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:27.718283 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa72532-6f6c-4463-a120-294b34e6bf24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:27.718958 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa72532-6f6c-4463-a120-294b34e6bf24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:27.720876 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa72532-6f6c-4463-a120-294b34e6bf24 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:27.747055 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1401/2790] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:29:27 2022] POST /identity/v3/projects => generated 373 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:29:27.751048 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc3b04-7ab9-4fce-be49-2db6351bc020 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:27.761504 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc3b04-7ab9-4fce-be49-2db6351bc020 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:27.816342 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a46fcb-c3b0-4a71-92b1-324e37d0807d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:27.816997 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a46fcb-c3b0-4a71-92b1-324e37d0807d admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:27.817281 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a46fcb-c3b0-4a71-92b1-324e37d0807d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:27.817557 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a46fcb-c3b0-4a71-92b1-324e37d0807d admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:27.818278 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a46fcb-c3b0-4a71-92b1-324e37d0807d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:27.820227 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a46fcb-c3b0-4a71-92b1-324e37d0807d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:27.838646 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1390/2791] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:29:27 2022] POST /identity/v3/users => generated 293 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:29:27.842461 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa72532-6f6c-4463-a120-294b34e6bf24 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:27.853245 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa72532-6f6c-4463-a120-294b34e6bf24 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:27.857873 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af558a-27c8-4543-8932-cff43b5bf502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:27.858537 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af558a-27c8-4543-8932-cff43b5bf502 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:27.858816 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af558a-27c8-4543-8932-cff43b5bf502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:27.859096 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af558a-27c8-4543-8932-cff43b5bf502 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:27.859857 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af558a-27c8-4543-8932-cff43b5bf502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:27.861436 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af558a-27c8-4543-8932-cff43b5bf502 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:27.871723 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1402/2792] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:29:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:29:27.875531 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a46fcb-c3b0-4a71-92b1-324e37d0807d admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:27.885270 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a46fcb-c3b0-4a71-92b1-324e37d0807d admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:27.889814 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c350f0ef-67f0-4a17-b8b5-b609e42095bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:27.890677 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350f0ef-67f0-4a17-b8b5-b609e42095bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:27.890960 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350f0ef-67f0-4a17-b8b5-b609e42095bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:27.891235 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350f0ef-67f0-4a17-b8b5-b609e42095bf admin admin] PATH_INFO: `/v3/projects/5cf591f965ee41eeb81fa3d04235b189/users/e3d0963552ee46a2a569117980420cf5/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:27.907847 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c350f0ef-67f0-4a17-b8b5-b609e42095bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cf591f965ee41eeb81fa3d04235b189, user_id=e3d0963552ee46a2a569117980420cf5, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:27.911074 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c350f0ef-67f0-4a17-b8b5-b609e42095bf admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:27.924944 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1391/2793] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:29:27 2022] PUT /identity/v3/projects/5cf591f965ee41eeb81fa3d04235b189/users/e3d0963552ee46a2a569117980420cf5/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:27.928905 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af558a-27c8-4543-8932-cff43b5bf502 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:27.938929 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af558a-27c8-4543-8932-cff43b5bf502 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:27.994535 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec7cef-a76d-43f2-a364-901bc04551b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:27.995207 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec7cef-a76d-43f2-a364-901bc04551b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:27.995527 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec7cef-a76d-43f2-a364-901bc04551b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:27.995811 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec7cef-a76d-43f2-a364-901bc04551b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:27.996532 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ec7cef-a76d-43f2-a364-901bc04551b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:27.998117 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ec7cef-a76d-43f2-a364-901bc04551b5 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:28.008128 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1403/2794] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:29:27 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:29:28.012220 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c350f0ef-67f0-4a17-b8b5-b609e42095bf admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:28.022656 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c350f0ef-67f0-4a17-b8b5-b609e42095bf admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:28.027114 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb701828-2263-4f04-8951-ece5ecd85bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:28.028284 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb701828-2263-4f04-8951-ece5ecd85bad admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:28.028567 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb701828-2263-4f04-8951-ece5ecd85bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:28.028844 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb701828-2263-4f04-8951-ece5ecd85bad admin admin] PATH_INFO: `/v3/projects/5cf591f965ee41eeb81fa3d04235b189/users/e3d0963552ee46a2a569117980420cf5/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:28.030773 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb701828-2263-4f04-8951-ece5ecd85bad admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cf591f965ee41eeb81fa3d04235b189, user_id=e3d0963552ee46a2a569117980420cf5, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:28.033391 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb701828-2263-4f04-8951-ece5ecd85bad admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:28.039528 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:28.045030 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1392/2795] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:29:28 2022] PUT /identity/v3/projects/5cf591f965ee41eeb81fa3d04235b189/users/e3d0963552ee46a2a569117980420cf5/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:29:28.054917 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082da20b-1570-4d5c-bc10-af95ce2534fd None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:28.055213 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082da20b-1570-4d5c-bc10-af95ce2534fd None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:28.055557 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082da20b-1570-4d5c-bc10-af95ce2534fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:28.068117 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:28.082677 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-082da20b-1570-4d5c-bc10-af95ce2534fd None None] MFA Rules not processed for user `e3d0963552ee46a2a569117980420cf5`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:29:28.083908 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-082da20b-1570-4d5c-bc10-af95ce2534fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:28.152710 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1404/2796] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:29:28 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:29:28.164079 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb701828-2263-4f04-8951-ece5ecd85bad admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:28.165821 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-cb701828-2263-4f04-8951-ece5ecd85bad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:28.221235 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb701828-2263-4f04-8951-ece5ecd85bad admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:28.224969 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb70120-9228-46f3-9c41-d856c2442a99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:28.225614 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb70120-9228-46f3-9c41-d856c2442a99 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:28.225903 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb70120-9228-46f3-9c41-d856c2442a99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:28.226187 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb70120-9228-46f3-9c41-d856c2442a99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:28.236840 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb70120-9228-46f3-9c41-d856c2442a99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:28.238649 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb70120-9228-46f3-9c41-d856c2442a99 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:28.248432 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1393/2797] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:29:28 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:29:28.639247 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082da20b-1570-4d5c-bc10-af95ce2534fd None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:28.702638 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082da20b-1570-4d5c-bc10-af95ce2534fd None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:28.707098 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc4f58-b639-4b34-9e59-fe950fb335a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:28.707994 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc4f58-b639-4b34-9e59-fe950fb335a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:28.708278 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc4f58-b639-4b34-9e59-fe950fb335a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:28.708555 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc4f58-b639-4b34-9e59-fe950fb335a8 admin admin] PATH_INFO: `/v3/users/fd9f6984e1764a1abe892540f8d0459b` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:28.709777 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cc4f58-b639-4b34-9e59-fe950fb335a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd9f6984e1764a1abe892540f8d0459b)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:28.711726 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cc4f58-b639-4b34-9e59-fe950fb335a8 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:28.757021 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-95cc4f58-b639-4b34-9e59-fe950fb335a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd9f6984e1764a1abe892540f8d0459b'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:28.763870 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-95cc4f58-b639-4b34-9e59-fe950fb335a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd9f6984e1764a1abe892540f8d0459b'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:28.776071 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-95cc4f58-b639-4b34-9e59-fe950fb335a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd9f6984e1764a1abe892540f8d0459b'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:28.783128 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-95cc4f58-b639-4b34-9e59-fe950fb335a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd9f6984e1764a1abe892540f8d0459b'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:28.783810 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-95cc4f58-b639-4b34-9e59-fe950fb335a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd9f6984e1764a1abe892540f8d0459b'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:28.790252 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1405/2798] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:29:28 2022] DELETE /identity/v3/users/fd9f6984e1764a1abe892540f8d0459b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:28.794739 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe353c-12f7-4198-91dd-b839832dd36a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:28.795223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe353c-12f7-4198-91dd-b839832dd36a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:28.795371 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe353c-12f7-4198-91dd-b839832dd36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:28.812891 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-81fe353c-12f7-4198-91dd-b839832dd36a None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:29:28.814076 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-81fe353c-12f7-4198-91dd-b839832dd36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:28.870839 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1394/2799] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:29:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:29:28.880902 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc4f58-b639-4b34-9e59-fe950fb335a8 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:28.882334 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-95cc4f58-b639-4b34-9e59-fe950fb335a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:28.897800 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:28.943109 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc4f58-b639-4b34-9e59-fe950fb335a8 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:28.946788 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c7c2f6-50c8-4e6c-aed9-4f09e57b0295 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:28.947468 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7c2f6-50c8-4e6c-aed9-4f09e57b0295 service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:28.947759 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7c2f6-50c8-4e6c-aed9-4f09e57b0295 service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:28.948041 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7c2f6-50c8-4e6c-aed9-4f09e57b0295 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:28.957691 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c7c2f6-50c8-4e6c-aed9-4f09e57b0295 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:28.959049 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c7c2f6-50c8-4e6c-aed9-4f09e57b0295 service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:28.968843 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1406/2800] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:29:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:29:29.232702 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe353c-12f7-4198-91dd-b839832dd36a None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:29.234673 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-81fe353c-12f7-4198-91dd-b839832dd36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:29.249526 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe353c-12f7-4198-91dd-b839832dd36a None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:29.255511 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa6d5d5-3a41-40b3-bdcf-f16d0c010c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:29.256458 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6d5d5-3a41-40b3-bdcf-f16d0c010c15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:29.256811 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6d5d5-3a41-40b3-bdcf-f16d0c010c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:29.257163 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6d5d5-3a41-40b3-bdcf-f16d0c010c15 admin admin] PATH_INFO: `/v3/projects/a1f1969c4ac947e6aec774ee02c283dd` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:29.258462 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa6d5d5-3a41-40b3-bdcf-f16d0c010c15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1f1969c4ac947e6aec774ee02c283dd)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:29.260519 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa6d5d5-3a41-40b3-bdcf-f16d0c010c15 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:29.307015 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4aa6d5d5-3a41-40b3-bdcf-f16d0c010c15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1f1969c4ac947e6aec774ee02c283dd', 'request_id': 'req-4aa6d5d5-3a41-40b3-bdcf-f16d0c010c15'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:29.313060 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4aa6d5d5-3a41-40b3-bdcf-f16d0c010c15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1f1969c4ac947e6aec774ee02c283dd', 'request_id': 'req-4aa6d5d5-3a41-40b3-bdcf-f16d0c010c15'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:29.320935 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4aa6d5d5-3a41-40b3-bdcf-f16d0c010c15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1f1969c4ac947e6aec774ee02c283dd', 'request_id': 'req-4aa6d5d5-3a41-40b3-bdcf-f16d0c010c15'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:29.331138 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4aa6d5d5-3a41-40b3-bdcf-f16d0c010c15 admin admin] The token cache is being invalidate because project a1f1969c4ac947e6aec774ee02c283dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:29:29.331483 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4aa6d5d5-3a41-40b3-bdcf-f16d0c010c15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:29.332126 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4aa6d5d5-3a41-40b3-bdcf-f16d0c010c15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:29.333635 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1395/2801] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:29:29 2022] DELETE /identity/v3/projects/a1f1969c4ac947e6aec774ee02c283dd => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:29.337514 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c7c2f6-50c8-4e6c-aed9-4f09e57b0295 service neutron] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:29.347826 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c7c2f6-50c8-4e6c-aed9-4f09e57b0295 service neutron] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:29.404776 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c25be7-f3a3-49f6-bb3c-282cc95d8fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:29.405657 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c25be7-f3a3-49f6-bb3c-282cc95d8fd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:29.405946 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c25be7-f3a3-49f6-bb3c-282cc95d8fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:29.406229 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c25be7-f3a3-49f6-bb3c-282cc95d8fd4 admin admin] PATH_INFO: `/v3/users/e3d0963552ee46a2a569117980420cf5` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:29.407448 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c25be7-f3a3-49f6-bb3c-282cc95d8fd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3d0963552ee46a2a569117980420cf5)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:29.409347 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c25be7-f3a3-49f6-bb3c-282cc95d8fd4 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:29.454814 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-75c25be7-f3a3-49f6-bb3c-282cc95d8fd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3d0963552ee46a2a569117980420cf5'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:29.462856 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-75c25be7-f3a3-49f6-bb3c-282cc95d8fd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3d0963552ee46a2a569117980420cf5'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:29.474565 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-75c25be7-f3a3-49f6-bb3c-282cc95d8fd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3d0963552ee46a2a569117980420cf5'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:29.481302 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-75c25be7-f3a3-49f6-bb3c-282cc95d8fd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3d0963552ee46a2a569117980420cf5'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:29.481863 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-75c25be7-f3a3-49f6-bb3c-282cc95d8fd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3d0963552ee46a2a569117980420cf5'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:29.488196 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1407/2802] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:29:29 2022] DELETE /identity/v3/users/e3d0963552ee46a2a569117980420cf5 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:29.765557 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa6d5d5-3a41-40b3-bdcf-f16d0c010c15 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:29.767287 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4aa6d5d5-3a41-40b3-bdcf-f16d0c010c15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:29.783007 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa6d5d5-3a41-40b3-bdcf-f16d0c010c15 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:29.808826 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:29.849265 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e882f8ce-701e-4b56-a93e-b400008221b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:29.850242 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882f8ce-701e-4b56-a93e-b400008221b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:29.850532 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882f8ce-701e-4b56-a93e-b400008221b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:29.850816 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882f8ce-701e-4b56-a93e-b400008221b8 admin admin] PATH_INFO: `/v3/projects/5cf591f965ee41eeb81fa3d04235b189` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:29.852138 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e882f8ce-701e-4b56-a93e-b400008221b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cf591f965ee41eeb81fa3d04235b189)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:29.854409 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e882f8ce-701e-4b56-a93e-b400008221b8 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:29.902234 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e882f8ce-701e-4b56-a93e-b400008221b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cf591f965ee41eeb81fa3d04235b189', 'request_id': 'req-e882f8ce-701e-4b56-a93e-b400008221b8'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:29.908750 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e882f8ce-701e-4b56-a93e-b400008221b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cf591f965ee41eeb81fa3d04235b189', 'request_id': 'req-e882f8ce-701e-4b56-a93e-b400008221b8'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:29.918230 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e882f8ce-701e-4b56-a93e-b400008221b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cf591f965ee41eeb81fa3d04235b189', 'request_id': 'req-e882f8ce-701e-4b56-a93e-b400008221b8'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:29.929574 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e882f8ce-701e-4b56-a93e-b400008221b8 admin admin] The token cache is being invalidate because project 5cf591f965ee41eeb81fa3d04235b189 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:29:29.929871 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e882f8ce-701e-4b56-a93e-b400008221b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:29.930509 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e882f8ce-701e-4b56-a93e-b400008221b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:29.932082 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1396/2803] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:29:29 2022] DELETE /identity/v3/projects/5cf591f965ee41eeb81fa3d04235b189 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:29.941339 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd5611-cbf0-4f4b-8713-ea6151647f28 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:29.941894 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd5611-cbf0-4f4b-8713-ea6151647f28 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:29.941894 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd5611-cbf0-4f4b-8713-ea6151647f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:29.960943 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-32cd5611-cbf0-4f4b-8713-ea6151647f28 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:29:29.962141 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-32cd5611-cbf0-4f4b-8713-ea6151647f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:29.996239 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:30.030463 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1408/2804] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:29:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:29:30.038445 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9d322-8260-41a7-a857-e655569d8760 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:30.038984 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9d322-8260-41a7-a857-e655569d8760 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:30.039277 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9d322-8260-41a7-a857-e655569d8760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:30.057873 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-73f9d322-8260-41a7-a857-e655569d8760 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:29:30.059098 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-73f9d322-8260-41a7-a857-e655569d8760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:30.067140 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1397/2805] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:29:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:29:30.071463 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd5611-cbf0-4f4b-8713-ea6151647f28 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:30.082484 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd5611-cbf0-4f4b-8713-ea6151647f28 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:30.087185 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c6686-cfc5-4f8c-8599-6d891c9494f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:30.087898 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c6686-cfc5-4f8c-8599-6d891c9494f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:30.088188 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c6686-cfc5-4f8c-8599-6d891c9494f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:30.088470 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c6686-cfc5-4f8c-8599-6d891c9494f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:30.089240 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c6686-cfc5-4f8c-8599-6d891c9494f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:29:30.089554 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c6686-cfc5-4f8c-8599-6d891c9494f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:30.091180 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c6686-cfc5-4f8c-8599-6d891c9494f7 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:30.106143 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1409/2806] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:29:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:29:30.110547 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9d322-8260-41a7-a857-e655569d8760 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:30.119895 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9d322-8260-41a7-a857-e655569d8760 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:30.125873 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa1d590-d076-46c5-8901-4ce5e63fc0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:30.125873 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1d590-d076-46c5-8901-4ce5e63fc0e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:30.125873 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1d590-d076-46c5-8901-4ce5e63fc0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:30.125873 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1d590-d076-46c5-8901-4ce5e63fc0e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:30.127028 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa1d590-d076-46c5-8901-4ce5e63fc0e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:30.129497 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa1d590-d076-46c5-8901-4ce5e63fc0e3 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:30.160874 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1398/2807] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:29:30 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:29:30.161968 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c6686-cfc5-4f8c-8599-6d891c9494f7 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:30.172059 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c6686-cfc5-4f8c-8599-6d891c9494f7 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:30.233067 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506856fb-3244-44a0-a638-3a051ec3eb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:30.233752 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506856fb-3244-44a0-a638-3a051ec3eb1b admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:30.234028 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506856fb-3244-44a0-a638-3a051ec3eb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:30.234307 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506856fb-3244-44a0-a638-3a051ec3eb1b admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:30.235012 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506856fb-3244-44a0-a638-3a051ec3eb1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:30.237004 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506856fb-3244-44a0-a638-3a051ec3eb1b admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:30.263370 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1410/2808] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:29:30 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:29:30.266552 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa1d590-d076-46c5-8901-4ce5e63fc0e3 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:30.277852 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa1d590-d076-46c5-8901-4ce5e63fc0e3 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:30.282575 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf8940-1237-41d2-82cf-3caa31fadf92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:30.283236 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf8940-1237-41d2-82cf-3caa31fadf92 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:30.283614 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf8940-1237-41d2-82cf-3caa31fadf92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:30.284019 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf8940-1237-41d2-82cf-3caa31fadf92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:30.285281 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bf8940-1237-41d2-82cf-3caa31fadf92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:30.286883 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bf8940-1237-41d2-82cf-3caa31fadf92 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:30.299331 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1399/2809] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:29:30.302893 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506856fb-3244-44a0-a638-3a051ec3eb1b admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:30.313310 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506856fb-3244-44a0-a638-3a051ec3eb1b admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:30.318638 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3479614e-41c9-4ffa-bbb1-041d696f47ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:30.319472 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479614e-41c9-4ffa-bbb1-041d696f47ef admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:30.319759 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479614e-41c9-4ffa-bbb1-041d696f47ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:30.320034 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479614e-41c9-4ffa-bbb1-041d696f47ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:30.320787 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3479614e-41c9-4ffa-bbb1-041d696f47ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:30.322389 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3479614e-41c9-4ffa-bbb1-041d696f47ef admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:30.333215 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1411/2810] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:29:30.337845 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf8940-1237-41d2-82cf-3caa31fadf92 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:30.347546 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf8940-1237-41d2-82cf-3caa31fadf92 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:30.352134 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdef622-044b-4d48-9db6-bbfd1cc7fd94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:30.353021 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdef622-044b-4d48-9db6-bbfd1cc7fd94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:30.353292 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdef622-044b-4d48-9db6-bbfd1cc7fd94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:30.353573 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdef622-044b-4d48-9db6-bbfd1cc7fd94 admin admin] PATH_INFO: `/v3/projects/ef6e69e4a83044eb9a2b8676b0f4a157/users/0c0606f08a1a484b8bcc8daea01b09b9/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:30.369466 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdef622-044b-4d48-9db6-bbfd1cc7fd94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef6e69e4a83044eb9a2b8676b0f4a157, user_id=0c0606f08a1a484b8bcc8daea01b09b9, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:30.372384 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdef622-044b-4d48-9db6-bbfd1cc7fd94 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:30.387693 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1400/2811] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:29:30 2022] PUT /identity/v3/projects/ef6e69e4a83044eb9a2b8676b0f4a157/users/0c0606f08a1a484b8bcc8daea01b09b9/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:30.399060 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c27d3-60a4-4db1-b2c1-8db410b7081f None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:30.399463 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c27d3-60a4-4db1-b2c1-8db410b7081f None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:30.399878 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c27d3-60a4-4db1-b2c1-8db410b7081f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:30.426758 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-b46c27d3-60a4-4db1-b2c1-8db410b7081f None None] MFA Rules not processed for user `0c0606f08a1a484b8bcc8daea01b09b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:29:30.430800 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-b46c27d3-60a4-4db1-b2c1-8db410b7081f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:30.501491 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1412/2812] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:29:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:29:30.506349 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdef622-044b-4d48-9db6-bbfd1cc7fd94 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:30.561425 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:30.574983 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdef622-044b-4d48-9db6-bbfd1cc7fd94 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:30.580277 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb75c88-e007-4c34-b61f-1ded7a1e25ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:30.580994 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb75c88-e007-4c34-b61f-1ded7a1e25ed admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:30.581766 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb75c88-e007-4c34-b61f-1ded7a1e25ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:30.582039 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb75c88-e007-4c34-b61f-1ded7a1e25ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:30.582739 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb75c88-e007-4c34-b61f-1ded7a1e25ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:30.584661 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb75c88-e007-4c34-b61f-1ded7a1e25ed admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:30.614058 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1401/2813] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:29:30 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:29:30.615105 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c27d3-60a4-4db1-b2c1-8db410b7081f None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:30.625866 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c27d3-60a4-4db1-b2c1-8db410b7081f None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:30.652012 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:30.684294 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a1162-465e-4fbf-8149-8a124ad3edae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:30.684965 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a1162-465e-4fbf-8149-8a124ad3edae admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:30.685260 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a1162-465e-4fbf-8149-8a124ad3edae admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:30.685538 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a1162-465e-4fbf-8149-8a124ad3edae admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:30.686246 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a1162-465e-4fbf-8149-8a124ad3edae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:30.688236 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a1162-465e-4fbf-8149-8a124ad3edae admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:30.708725 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1413/2814] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:29:30 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:29:30.712881 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb75c88-e007-4c34-b61f-1ded7a1e25ed admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:30.723175 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb75c88-e007-4c34-b61f-1ded7a1e25ed admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:30.727861 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cc92b-eaa4-4e07-a23e-4ede8592f328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:30.728517 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cc92b-eaa4-4e07-a23e-4ede8592f328 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:30.728803 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cc92b-eaa4-4e07-a23e-4ede8592f328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:30.729082 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cc92b-eaa4-4e07-a23e-4ede8592f328 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:30.729804 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3cc92b-eaa4-4e07-a23e-4ede8592f328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:30.731414 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3cc92b-eaa4-4e07-a23e-4ede8592f328 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:30.742298 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1402/2815] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:29:30.746711 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a1162-465e-4fbf-8149-8a124ad3edae admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:30.758744 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a1162-465e-4fbf-8149-8a124ad3edae admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:30.764039 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b3344-a687-4e7c-829d-0d6fa6490a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:30.765004 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b3344-a687-4e7c-829d-0d6fa6490a83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:30.765289 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b3344-a687-4e7c-829d-0d6fa6490a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:30.765584 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b3344-a687-4e7c-829d-0d6fa6490a83 admin admin] PATH_INFO: `/v3/projects/c796666648a84d43897ba9439513ca8f/users/2ab6eb1b427248ce8d114e9e2284a078/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:30.784574 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792b3344-a687-4e7c-829d-0d6fa6490a83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c796666648a84d43897ba9439513ca8f, user_id=2ab6eb1b427248ce8d114e9e2284a078, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:30.787215 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792b3344-a687-4e7c-829d-0d6fa6490a83 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:30.800935 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1414/2816] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:29:30 2022] PUT /identity/v3/projects/c796666648a84d43897ba9439513ca8f/users/2ab6eb1b427248ce8d114e9e2284a078/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:29:30.805100 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cc92b-eaa4-4e07-a23e-4ede8592f328 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:30.815101 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cc92b-eaa4-4e07-a23e-4ede8592f328 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:30.871598 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29245d0-80d3-4cff-9fb0-5ea6c2fc7ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:30.872272 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29245d0-80d3-4cff-9fb0-5ea6c2fc7ecf admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:30.872563 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29245d0-80d3-4cff-9fb0-5ea6c2fc7ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:30.872847 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29245d0-80d3-4cff-9fb0-5ea6c2fc7ecf admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:30.873571 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29245d0-80d3-4cff-9fb0-5ea6c2fc7ecf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:30.875170 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29245d0-80d3-4cff-9fb0-5ea6c2fc7ecf admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:30.885504 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1403/2817] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:29:30.889747 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b3344-a687-4e7c-829d-0d6fa6490a83 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:30.899751 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b3344-a687-4e7c-829d-0d6fa6490a83 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:30.904502 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63c8aa-56a6-490a-931c-a4bfe0b24cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:30.905375 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63c8aa-56a6-490a-931c-a4bfe0b24cee admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:30.905655 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63c8aa-56a6-490a-931c-a4bfe0b24cee admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:30.905928 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63c8aa-56a6-490a-931c-a4bfe0b24cee admin admin] PATH_INFO: `/v3/projects/c796666648a84d43897ba9439513ca8f/users/2ab6eb1b427248ce8d114e9e2284a078/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:30.907877 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63c8aa-56a6-490a-931c-a4bfe0b24cee admin admin] RBAC: Authorizing `identity:create_grant(project_id=c796666648a84d43897ba9439513ca8f, user_id=2ab6eb1b427248ce8d114e9e2284a078, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:30.910451 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63c8aa-56a6-490a-931c-a4bfe0b24cee admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:30.920802 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1415/2818] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:29:30 2022] PUT /identity/v3/projects/c796666648a84d43897ba9439513ca8f/users/2ab6eb1b427248ce8d114e9e2284a078/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:29:30.932112 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3266fd-554a-48ea-a433-f348520fdabd None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:30.932404 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3266fd-554a-48ea-a433-f348520fdabd None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:30.932686 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3266fd-554a-48ea-a433-f348520fdabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:30.958816 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-bc3266fd-554a-48ea-a433-f348520fdabd None None] MFA Rules not processed for user `2ab6eb1b427248ce8d114e9e2284a078`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:29:30.960081 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-bc3266fd-554a-48ea-a433-f348520fdabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:31.026936 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1404/2819] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:29:30 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:29:31.038669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63c8aa-56a6-490a-931c-a4bfe0b24cee admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:31.040416 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-1b63c8aa-56a6-490a-931c-a4bfe0b24cee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:31.095225 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63c8aa-56a6-490a-931c-a4bfe0b24cee admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:31.099372 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c38d4c-f9fc-44aa-9e92-68e09b666c27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:31.100000 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c38d4c-f9fc-44aa-9e92-68e09b666c27 service cinder] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:31.100281 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c38d4c-f9fc-44aa-9e92-68e09b666c27 service cinder] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:31.100556 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c38d4c-f9fc-44aa-9e92-68e09b666c27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:31.109223 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c38d4c-f9fc-44aa-9e92-68e09b666c27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:31.110646 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c38d4c-f9fc-44aa-9e92-68e09b666c27 service cinder] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:31.120401 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1416/2820] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:29:31 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:29:31.186595 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3266fd-554a-48ea-a433-f348520fdabd None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:31.233605 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:31.249669 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3266fd-554a-48ea-a433-f348520fdabd None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:31.254372 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64eca1a-5d4f-4f97-a00b-9c0cc42a2d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:31.255280 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64eca1a-5d4f-4f97-a00b-9c0cc42a2d15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:31.255596 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64eca1a-5d4f-4f97-a00b-9c0cc42a2d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:31.255877 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64eca1a-5d4f-4f97-a00b-9c0cc42a2d15 admin admin] PATH_INFO: `/v3/users/0c0606f08a1a484b8bcc8daea01b09b9` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:31.257089 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64eca1a-5d4f-4f97-a00b-9c0cc42a2d15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c0606f08a1a484b8bcc8daea01b09b9)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:31.259012 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64eca1a-5d4f-4f97-a00b-9c0cc42a2d15 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:31.304974 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c64eca1a-5d4f-4f97-a00b-9c0cc42a2d15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c0606f08a1a484b8bcc8daea01b09b9'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:31.311988 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c64eca1a-5d4f-4f97-a00b-9c0cc42a2d15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c0606f08a1a484b8bcc8daea01b09b9'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:31.323833 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c64eca1a-5d4f-4f97-a00b-9c0cc42a2d15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c0606f08a1a484b8bcc8daea01b09b9'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:31.324471 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c64eca1a-5d4f-4f97-a00b-9c0cc42a2d15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c0606f08a1a484b8bcc8daea01b09b9'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:31.331098 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c64eca1a-5d4f-4f97-a00b-9c0cc42a2d15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c0606f08a1a484b8bcc8daea01b09b9'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:31.336872 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1405/2821] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:29:31 2022] DELETE /identity/v3/users/0c0606f08a1a484b8bcc8daea01b09b9 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:31.341302 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac09f22-287c-41af-8186-d47614ccfdaa None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:31.341589 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac09f22-287c-41af-8186-d47614ccfdaa None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:31.341865 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac09f22-287c-41af-8186-d47614ccfdaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:31.359443 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-2ac09f22-287c-41af-8186-d47614ccfdaa None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:29:31.360622 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-2ac09f22-287c-41af-8186-d47614ccfdaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:31.418298 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1417/2822] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:29:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:29:31.428422 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64eca1a-5d4f-4f97-a00b-9c0cc42a2d15 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:31.429834 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c64eca1a-5d4f-4f97-a00b-9c0cc42a2d15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:31.486353 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64eca1a-5d4f-4f97-a00b-9c0cc42a2d15 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:31.490044 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51d5a0-3122-4e18-9068-41548173aab7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:31.490709 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51d5a0-3122-4e18-9068-41548173aab7 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:31.490997 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51d5a0-3122-4e18-9068-41548173aab7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:31.491276 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51d5a0-3122-4e18-9068-41548173aab7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:31.500219 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c51d5a0-3122-4e18-9068-41548173aab7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:31.501640 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c51d5a0-3122-4e18-9068-41548173aab7 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:31.511714 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1406/2823] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:29:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:29:31.789774 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac09f22-287c-41af-8186-d47614ccfdaa None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:31.791673 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-2ac09f22-287c-41af-8186-d47614ccfdaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:31.794816 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51d5a0-3122-4e18-9068-41548173aab7 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:31.796310 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0c51d5a0-3122-4e18-9068-41548173aab7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:31.804347 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac09f22-287c-41af-8186-d47614ccfdaa None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:31.809432 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e8c7d-b3de-4639-b308-8a33ef8cb768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:31.810394 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e8c7d-b3de-4639-b308-8a33ef8cb768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:31.810540 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51d5a0-3122-4e18-9068-41548173aab7 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:31.810749 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e8c7d-b3de-4639-b308-8a33ef8cb768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:31.810978 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e8c7d-b3de-4639-b308-8a33ef8cb768 admin admin] PATH_INFO: `/v3/projects/ef6e69e4a83044eb9a2b8676b0f4a157` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:31.812265 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e8c7d-b3de-4639-b308-8a33ef8cb768 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef6e69e4a83044eb9a2b8676b0f4a157)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:31.814172 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e8c7d-b3de-4639-b308-8a33ef8cb768 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:31.816561 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22703d4-29b6-4b7c-b103-7ce83868f50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:31.817247 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22703d4-29b6-4b7c-b103-7ce83868f50c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:31.817533 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22703d4-29b6-4b7c-b103-7ce83868f50c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:31.817811 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22703d4-29b6-4b7c-b103-7ce83868f50c admin admin] PATH_INFO: `/v3/users/3853921a5a414013be902cfd984b9248` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:31.819055 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22703d4-29b6-4b7c-b103-7ce83868f50c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3853921a5a414013be902cfd984b9248)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:31.821002 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22703d4-29b6-4b7c-b103-7ce83868f50c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:31.838134 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:31.867857 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-8a6e8c7d-b3de-4639-b308-8a33ef8cb768 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef6e69e4a83044eb9a2b8676b0f4a157', 'request_id': 'req-8a6e8c7d-b3de-4639-b308-8a33ef8cb768'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:31.869943 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b22703d4-29b6-4b7c-b103-7ce83868f50c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3853921a5a414013be902cfd984b9248'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:31.874210 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-8a6e8c7d-b3de-4639-b308-8a33ef8cb768 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef6e69e4a83044eb9a2b8676b0f4a157', 'request_id': 'req-8a6e8c7d-b3de-4639-b308-8a33ef8cb768'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:31.879851 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b22703d4-29b6-4b7c-b103-7ce83868f50c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3853921a5a414013be902cfd984b9248'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:31.883387 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-8a6e8c7d-b3de-4639-b308-8a33ef8cb768 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef6e69e4a83044eb9a2b8676b0f4a157', 'request_id': 'req-8a6e8c7d-b3de-4639-b308-8a33ef8cb768'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:31.887037 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:31.894778 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-8a6e8c7d-b3de-4639-b308-8a33ef8cb768 admin admin] The token cache is being invalidate because project ef6e69e4a83044eb9a2b8676b0f4a157 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:29:31.895070 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-8a6e8c7d-b3de-4639-b308-8a33ef8cb768 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:31.895840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-8a6e8c7d-b3de-4639-b308-8a33ef8cb768 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:31.896664 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b22703d4-29b6-4b7c-b103-7ce83868f50c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3853921a5a414013be902cfd984b9248'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:31.897411 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1418/2824] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:29:31 2022] DELETE /identity/v3/projects/ef6e69e4a83044eb9a2b8676b0f4a157 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:31.897539 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b22703d4-29b6-4b7c-b103-7ce83868f50c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3853921a5a414013be902cfd984b9248'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:31.901327 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e8c7d-b3de-4639-b308-8a33ef8cb768 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:31.902880 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-8a6e8c7d-b3de-4639-b308-8a33ef8cb768 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:31.905692 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b22703d4-29b6-4b7c-b103-7ce83868f50c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3853921a5a414013be902cfd984b9248'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:31.912019 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1407/2825] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:29:31 2022] DELETE /identity/v3/users/3853921a5a414013be902cfd984b9248 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:29:31.916303 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1682ee02-fb89-4ac1-be34-fbf595e79e63 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:31.916598 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1682ee02-fb89-4ac1-be34-fbf595e79e63 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:31.916910 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1682ee02-fb89-4ac1-be34-fbf595e79e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:31.934165 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-1682ee02-fb89-4ac1-be34-fbf595e79e63 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:29:31.935420 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1682ee02-fb89-4ac1-be34-fbf595e79e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:31.968376 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e8c7d-b3de-4639-b308-8a33ef8cb768 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:31.995851 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1408/2826] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:29:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:29:32.005991 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1682ee02-fb89-4ac1-be34-fbf595e79e63 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:32.007354 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1682ee02-fb89-4ac1-be34-fbf595e79e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:32.023453 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f30848-88ca-4ebb-a85f-9992afb6ce24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:32.024360 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f30848-88ca-4ebb-a85f-9992afb6ce24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:32.024644 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f30848-88ca-4ebb-a85f-9992afb6ce24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:32.024923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f30848-88ca-4ebb-a85f-9992afb6ce24 admin admin] PATH_INFO: `/v3/users/2ab6eb1b427248ce8d114e9e2284a078` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:32.026135 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f30848-88ca-4ebb-a85f-9992afb6ce24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ab6eb1b427248ce8d114e9e2284a078)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:32.028383 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f30848-88ca-4ebb-a85f-9992afb6ce24 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:32.062777 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1682ee02-fb89-4ac1-be34-fbf595e79e63 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:32.067427 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1cd5d9-e5a7-4c0e-9261-4fb0ccad0912 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:32.068037 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1cd5d9-e5a7-4c0e-9261-4fb0ccad0912 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:32.068322 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1cd5d9-e5a7-4c0e-9261-4fb0ccad0912 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:32.068607 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1cd5d9-e5a7-4c0e-9261-4fb0ccad0912 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:32.073155 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-74f30848-88ca-4ebb-a85f-9992afb6ce24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ab6eb1b427248ce8d114e9e2284a078'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:32.078055 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1cd5d9-e5a7-4c0e-9261-4fb0ccad0912 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:32.080139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1cd5d9-e5a7-4c0e-9261-4fb0ccad0912 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:32.082251 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-74f30848-88ca-4ebb-a85f-9992afb6ce24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ab6eb1b427248ce8d114e9e2284a078'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:32.084179 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:32.091639 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1409/2827] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:29:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 16:29:32.095878 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-74f30848-88ca-4ebb-a85f-9992afb6ce24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ab6eb1b427248ce8d114e9e2284a078'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:32.103414 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-74f30848-88ca-4ebb-a85f-9992afb6ce24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ab6eb1b427248ce8d114e9e2284a078'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:32.104131 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-74f30848-88ca-4ebb-a85f-9992afb6ce24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ab6eb1b427248ce8d114e9e2284a078'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:32.110080 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1419/2828] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:29:31 2022] DELETE /identity/v3/users/2ab6eb1b427248ce8d114e9e2284a078 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:32.461607 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1cd5d9-e5a7-4c0e-9261-4fb0ccad0912 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:32.463491 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ac1cd5d9-e5a7-4c0e-9261-4fb0ccad0912 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:32.467768 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f30848-88ca-4ebb-a85f-9992afb6ce24 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:32.469233 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-74f30848-88ca-4ebb-a85f-9992afb6ce24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:32.481346 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1cd5d9-e5a7-4c0e-9261-4fb0ccad0912 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:32.543257 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47183d5-b0d8-425a-9864-ae83402f9124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:32.544158 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47183d5-b0d8-425a-9864-ae83402f9124 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:32.544444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47183d5-b0d8-425a-9864-ae83402f9124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:32.544723 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47183d5-b0d8-425a-9864-ae83402f9124 admin admin] PATH_INFO: `/v3/projects/45026d08db0c4a82a177078744d32801` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:32.545939 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47183d5-b0d8-425a-9864-ae83402f9124 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45026d08db0c4a82a177078744d32801)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:32.546185 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f30848-88ca-4ebb-a85f-9992afb6ce24 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:32.547902 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47183d5-b0d8-425a-9864-ae83402f9124 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:32.551251 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e1925-9c9e-468b-9dc7-70848584deca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:32.552140 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e1925-9c9e-468b-9dc7-70848584deca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:32.552428 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e1925-9c9e-468b-9dc7-70848584deca admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:32.552705 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e1925-9c9e-468b-9dc7-70848584deca admin admin] PATH_INFO: `/v3/projects/c796666648a84d43897ba9439513ca8f` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:32.553881 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636e1925-9c9e-468b-9dc7-70848584deca admin admin] RBAC: Authorizing `identity:delete_project(project_id=c796666648a84d43897ba9439513ca8f)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:32.555797 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636e1925-9c9e-468b-9dc7-70848584deca admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:32.578956 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:32.597260 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b47183d5-b0d8-425a-9864-ae83402f9124 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45026d08db0c4a82a177078744d32801', 'request_id': 'req-b47183d5-b0d8-425a-9864-ae83402f9124'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:32.602193 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-636e1925-9c9e-468b-9dc7-70848584deca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c796666648a84d43897ba9439513ca8f', 'request_id': 'req-636e1925-9c9e-468b-9dc7-70848584deca'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:32.603258 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b47183d5-b0d8-425a-9864-ae83402f9124 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45026d08db0c4a82a177078744d32801', 'request_id': 'req-b47183d5-b0d8-425a-9864-ae83402f9124'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:32.608902 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-636e1925-9c9e-468b-9dc7-70848584deca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c796666648a84d43897ba9439513ca8f', 'request_id': 'req-636e1925-9c9e-468b-9dc7-70848584deca'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:32.610716 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b47183d5-b0d8-425a-9864-ae83402f9124 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45026d08db0c4a82a177078744d32801', 'request_id': 'req-b47183d5-b0d8-425a-9864-ae83402f9124'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:32.616220 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-636e1925-9c9e-468b-9dc7-70848584deca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c796666648a84d43897ba9439513ca8f', 'request_id': 'req-636e1925-9c9e-468b-9dc7-70848584deca'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:32.622446 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b47183d5-b0d8-425a-9864-ae83402f9124 admin admin] The token cache is being invalidate because project 45026d08db0c4a82a177078744d32801 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:29:32.622737 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b47183d5-b0d8-425a-9864-ae83402f9124 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:32.623387 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b47183d5-b0d8-425a-9864-ae83402f9124 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:32.624955 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1410/2829] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:29:32 2022] DELETE /identity/v3/projects/45026d08db0c4a82a177078744d32801 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:32.627347 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-636e1925-9c9e-468b-9dc7-70848584deca admin admin] The token cache is being invalidate because project c796666648a84d43897ba9439513ca8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:29:32.627648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-636e1925-9c9e-468b-9dc7-70848584deca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:32.628583 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-636e1925-9c9e-468b-9dc7-70848584deca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:32.628903 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47183d5-b0d8-425a-9864-ae83402f9124 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:32.630165 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b47183d5-b0d8-425a-9864-ae83402f9124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:32.630296 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1420/2830] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:29:32 2022] DELETE /identity/v3/projects/c796666648a84d43897ba9439513ca8f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:29:32.636152 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5511f-c8f5-4b55-9f7f-f64933578720 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:32.636514 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5511f-c8f5-4b55-9f7f-f64933578720 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:32.636831 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5511f-c8f5-4b55-9f7f-f64933578720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:32.654872 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-5cc5511f-c8f5-4b55-9f7f-f64933578720 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:29:32.656113 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-5cc5511f-c8f5-4b55-9f7f-f64933578720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:32.693150 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47183d5-b0d8-425a-9864-ae83402f9124 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:32.713459 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1421/2831] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:29:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:29:32.720727 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4df0e-89c4-45f9-836a-5554eb1771ce None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:32.721069 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4df0e-89c4-45f9-836a-5554eb1771ce None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:32.721349 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4df0e-89c4-45f9-836a-5554eb1771ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:32.728745 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:32.742295 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-c9f4df0e-89c4-45f9-836a-5554eb1771ce None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:29:32.743520 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-c9f4df0e-89c4-45f9-836a-5554eb1771ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:32.751676 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1422/2832] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:29:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:29:32.752428 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6264bd-f646-4021-9719-eda91d9a12be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:32.753518 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6264bd-f646-4021-9719-eda91d9a12be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:32.753936 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6264bd-f646-4021-9719-eda91d9a12be admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:32.754340 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6264bd-f646-4021-9719-eda91d9a12be admin admin] PATH_INFO: `/v3/users/0fcb7cf4b48a41fa92e924c9d80e93b5` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:32.755657 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f4df0e-89c4-45f9-836a-5554eb1771ce None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:32.755975 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6264bd-f646-4021-9719-eda91d9a12be admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fcb7cf4b48a41fa92e924c9d80e93b5)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:32.758005 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6264bd-f646-4021-9719-eda91d9a12be admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:32.764278 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f4df0e-89c4-45f9-836a-5554eb1771ce None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:32.769075 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ffded1-22cc-4cb8-a555-bf83fcf3867d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:32.769713 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffded1-22cc-4cb8-a555-bf83fcf3867d admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:32.769994 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffded1-22cc-4cb8-a555-bf83fcf3867d admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:32.770270 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffded1-22cc-4cb8-a555-bf83fcf3867d admin admin] PATH_INFO: `/v3/domains` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:32.771021 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ffded1-22cc-4cb8-a555-bf83fcf3867d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80286) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:29:32.771353 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ffded1-22cc-4cb8-a555-bf83fcf3867d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:32.772928 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ffded1-22cc-4cb8-a555-bf83fcf3867d admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:32.788575 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1423/2833] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:29:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:29:32.789598 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ffded1-22cc-4cb8-a555-bf83fcf3867d admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:32.799557 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ffded1-22cc-4cb8-a555-bf83fcf3867d admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:32.804924 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb11a22-54b9-4c64-b94d-5ef3013a6433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:32.805557 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb11a22-54b9-4c64-b94d-5ef3013a6433 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:32.805685 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1b6264bd-f646-4021-9719-eda91d9a12be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fcb7cf4b48a41fa92e924c9d80e93b5'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:32.805844 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb11a22-54b9-4c64-b94d-5ef3013a6433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:32.806122 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb11a22-54b9-4c64-b94d-5ef3013a6433 admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:32.806816 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb11a22-54b9-4c64-b94d-5ef3013a6433 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:32.808719 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb11a22-54b9-4c64-b94d-5ef3013a6433 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:32.814529 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1b6264bd-f646-4021-9719-eda91d9a12be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fcb7cf4b48a41fa92e924c9d80e93b5'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:32.821969 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:32.827833 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1b6264bd-f646-4021-9719-eda91d9a12be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fcb7cf4b48a41fa92e924c9d80e93b5'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:32.828763 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1b6264bd-f646-4021-9719-eda91d9a12be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fcb7cf4b48a41fa92e924c9d80e93b5'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:32.839469 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1424/2834] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:29:32 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:29:32.839627 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1b6264bd-f646-4021-9719-eda91d9a12be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fcb7cf4b48a41fa92e924c9d80e93b5'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:32.840446 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb11a22-54b9-4c64-b94d-5ef3013a6433 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:32.842094 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-5fb11a22-54b9-4c64-b94d-5ef3013a6433 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:32.844785 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1411/2835] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:29:32 2022] DELETE /identity/v3/users/0fcb7cf4b48a41fa92e924c9d80e93b5 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:29:32.906690 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb11a22-54b9-4c64-b94d-5ef3013a6433 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:32.965669 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf2d20-0f37-4600-9a58-ce224eb1234e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:32.966352 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf2d20-0f37-4600-9a58-ce224eb1234e admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:32.966664 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf2d20-0f37-4600-9a58-ce224eb1234e admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:32.966941 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf2d20-0f37-4600-9a58-ce224eb1234e admin admin] PATH_INFO: `/v3/users` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:32.967672 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf2d20-0f37-4600-9a58-ce224eb1234e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:32.969593 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf2d20-0f37-4600-9a58-ce224eb1234e admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:32.992800 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1425/2836] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:29:32 2022] POST /identity/v3/users => generated 287 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 16:29:32.996870 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6264bd-f646-4021-9719-eda91d9a12be admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:33.007763 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6264bd-f646-4021-9719-eda91d9a12be admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:33.013030 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2bb82-3682-4cd2-b18c-056513bb4cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:33.013697 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2bb82-3682-4cd2-b18c-056513bb4cbe admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:33.013983 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2bb82-3682-4cd2-b18c-056513bb4cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:33.014259 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2bb82-3682-4cd2-b18c-056513bb4cbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:33.014992 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f2bb82-3682-4cd2-b18c-056513bb4cbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:33.016672 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f2bb82-3682-4cd2-b18c-056513bb4cbe admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:33.029602 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1412/2837] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:29:33.031577 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf2d20-0f37-4600-9a58-ce224eb1234e admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:33.042567 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf2d20-0f37-4600-9a58-ce224eb1234e admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:33.047289 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4594e9a8-354a-4763-af12-d61ce0176f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:33.047977 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594e9a8-354a-4763-af12-d61ce0176f9f admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:33.048269 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594e9a8-354a-4763-af12-d61ce0176f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:33.048545 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594e9a8-354a-4763-af12-d61ce0176f9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:33.049279 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4594e9a8-354a-4763-af12-d61ce0176f9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:33.050892 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4594e9a8-354a-4763-af12-d61ce0176f9f admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:33.060912 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1426/2838] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:29:33.065145 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2bb82-3682-4cd2-b18c-056513bb4cbe admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:33.075214 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2bb82-3682-4cd2-b18c-056513bb4cbe admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:33.079912 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75535d77-ab24-4ca7-95d0-3e6e47decc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:33.080793 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75535d77-ab24-4ca7-95d0-3e6e47decc40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:33.081611 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75535d77-ab24-4ca7-95d0-3e6e47decc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:33.081892 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75535d77-ab24-4ca7-95d0-3e6e47decc40 admin admin] PATH_INFO: `/v3/projects/896eaf64068542099d21437fb023e734/users/865b66dd2b0f4ac8822ffaf219ad3a77/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:33.097560 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75535d77-ab24-4ca7-95d0-3e6e47decc40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=896eaf64068542099d21437fb023e734, user_id=865b66dd2b0f4ac8822ffaf219ad3a77, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:33.100304 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75535d77-ab24-4ca7-95d0-3e6e47decc40 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:33.115510 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1413/2839] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:29:33 2022] PUT /identity/v3/projects/896eaf64068542099d21437fb023e734/users/865b66dd2b0f4ac8822ffaf219ad3a77/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:33.120488 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820c47a-30bc-4f34-abf1-2ea6a04bca3d None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:33.120939 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820c47a-30bc-4f34-abf1-2ea6a04bca3d None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:33.121061 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820c47a-30bc-4f34-abf1-2ea6a04bca3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:33.126456 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75535d77-ab24-4ca7-95d0-3e6e47decc40 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:33.128667 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-75535d77-ab24-4ca7-95d0-3e6e47decc40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:33.140108 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-7820c47a-30bc-4f34-abf1-2ea6a04bca3d None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:29:33.141421 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-7820c47a-30bc-4f34-abf1-2ea6a04bca3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:33.192507 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:33.198170 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75535d77-ab24-4ca7-95d0-3e6e47decc40 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:33.203419 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb7cdf-2461-4574-b0b1-89dba7f7498b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:33.204294 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb7cdf-2461-4574-b0b1-89dba7f7498b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:33.204578 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb7cdf-2461-4574-b0b1-89dba7f7498b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:33.204856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb7cdf-2461-4574-b0b1-89dba7f7498b admin admin] PATH_INFO: `/v3/projects/a2eb893f6ff9410ab3d72def028f0495` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:33.206559 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb7cdf-2461-4574-b0b1-89dba7f7498b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2eb893f6ff9410ab3d72def028f0495)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:33.208390 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1427/2840] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:29:33.208564 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb7cdf-2461-4574-b0b1-89dba7f7498b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:33.217856 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820c47a-30bc-4f34-abf1-2ea6a04bca3d None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:33.219252 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-7820c47a-30bc-4f34-abf1-2ea6a04bca3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:33.255679 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-bbbb7cdf-2461-4574-b0b1-89dba7f7498b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2eb893f6ff9410ab3d72def028f0495', 'request_id': 'req-bbbb7cdf-2461-4574-b0b1-89dba7f7498b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:33.261817 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-bbbb7cdf-2461-4574-b0b1-89dba7f7498b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2eb893f6ff9410ab3d72def028f0495', 'request_id': 'req-bbbb7cdf-2461-4574-b0b1-89dba7f7498b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:33.270589 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-bbbb7cdf-2461-4574-b0b1-89dba7f7498b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2eb893f6ff9410ab3d72def028f0495', 'request_id': 'req-bbbb7cdf-2461-4574-b0b1-89dba7f7498b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:33.281320 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-bbbb7cdf-2461-4574-b0b1-89dba7f7498b admin admin] The token cache is being invalidate because project a2eb893f6ff9410ab3d72def028f0495 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:29:33.281700 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-bbbb7cdf-2461-4574-b0b1-89dba7f7498b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:33.282420 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-bbbb7cdf-2461-4574-b0b1-89dba7f7498b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:33.284802 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1414/2841] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:29:33 2022] DELETE /identity/v3/projects/a2eb893f6ff9410ab3d72def028f0495 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:29:33.291864 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48d2bc-ff29-422a-bea0-6d2f14a3fe8c None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:33.292161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48d2bc-ff29-422a-bea0-6d2f14a3fe8c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:33.292450 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48d2bc-ff29-422a-bea0-6d2f14a3fe8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:33.296399 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820c47a-30bc-4f34-abf1-2ea6a04bca3d None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:33.311020 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-ee48d2bc-ff29-422a-bea0-6d2f14a3fe8c None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:29:33.312283 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ee48d2bc-ff29-422a-bea0-6d2f14a3fe8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:33.343799 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:33.350485 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7771d8c4-fc29-4eb3-8b4b-e7231e4c543a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:33.351125 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7771d8c4-fc29-4eb3-8b4b-e7231e4c543a service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:33.351445 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7771d8c4-fc29-4eb3-8b4b-e7231e4c543a service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:33.351724 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7771d8c4-fc29-4eb3-8b4b-e7231e4c543a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:33.354031 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-7771d8c4-fc29-4eb3-8b4b-e7231e4c543a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:33.373263 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1415/2842] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:29:33.381116 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5abd28-1946-43d3-bc58-00a70ac613e8 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:33.381403 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5abd28-1946-43d3-bc58-00a70ac613e8 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:33.381680 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5abd28-1946-43d3-bc58-00a70ac613e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:33.399789 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-4b5abd28-1946-43d3-bc58-00a70ac613e8 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:29:33.401000 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4b5abd28-1946-43d3-bc58-00a70ac613e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:33.409332 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1416/2843] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:29:33.413330 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5abd28-1946-43d3-bc58-00a70ac613e8 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:33.418556 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7771d8c4-fc29-4eb3-8b4b-e7231e4c543a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:33.420027 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7771d8c4-fc29-4eb3-8b4b-e7231e4c543a service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:33.422986 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5abd28-1946-43d3-bc58-00a70ac613e8 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:33.427797 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb07a679-de71-41c4-beb3-f18a79336aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:33.428431 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07a679-de71-41c4-beb3-f18a79336aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:33.428716 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07a679-de71-41c4-beb3-f18a79336aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:33.428995 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07a679-de71-41c4-beb3-f18a79336aa2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:33.429682 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1428/2844] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:29:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 16:29:33.429884 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb07a679-de71-41c4-beb3-f18a79336aa2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:29:33.430002 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb07a679-de71-41c4-beb3-f18a79336aa2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:33.431643 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb07a679-de71-41c4-beb3-f18a79336aa2 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:33.443918 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1417/2845] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:29:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 16:29:33.448775 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7771d8c4-fc29-4eb3-8b4b-e7231e4c543a service neutron] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:33.464982 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7771d8c4-fc29-4eb3-8b4b-e7231e4c543a service neutron] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:33.470172 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bdfcb6-d51f-4631-a00d-c56bad4a816d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:33.470902 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdfcb6-d51f-4631-a00d-c56bad4a816d admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:33.471212 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdfcb6-d51f-4631-a00d-c56bad4a816d admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:33.472053 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdfcb6-d51f-4631-a00d-c56bad4a816d admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:33.472852 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bdfcb6-d51f-4631-a00d-c56bad4a816d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:33.474867 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bdfcb6-d51f-4631-a00d-c56bad4a816d admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:33.501686 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1429/2846] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:29:33 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:29:33.506537 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb07a679-de71-41c4-beb3-f18a79336aa2 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:33.517245 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb07a679-de71-41c4-beb3-f18a79336aa2 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:33.582139 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd15286-e6c1-44ee-8e75-0f72c3de13fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:33.583081 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd15286-e6c1-44ee-8e75-0f72c3de13fc admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:33.583534 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd15286-e6c1-44ee-8e75-0f72c3de13fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:33.583944 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd15286-e6c1-44ee-8e75-0f72c3de13fc admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:33.584926 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd15286-e6c1-44ee-8e75-0f72c3de13fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:33.587565 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd15286-e6c1-44ee-8e75-0f72c3de13fc admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:33.608810 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1418/2847] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:29:33 2022] POST /identity/v3/users => generated 286 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:29:33.614392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bdfcb6-d51f-4631-a00d-c56bad4a816d admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:33.624793 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bdfcb6-d51f-4631-a00d-c56bad4a816d admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:33.629522 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44606b4f-d3ca-4f0a-a1cf-c7620f16065d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:33.630191 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44606b4f-d3ca-4f0a-a1cf-c7620f16065d admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:33.630482 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44606b4f-d3ca-4f0a-a1cf-c7620f16065d admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:33.630766 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44606b4f-d3ca-4f0a-a1cf-c7620f16065d admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:33.631534 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44606b4f-d3ca-4f0a-a1cf-c7620f16065d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:33.633154 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44606b4f-d3ca-4f0a-a1cf-c7620f16065d admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:33.644990 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1430/2848] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:29:33.649156 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd15286-e6c1-44ee-8e75-0f72c3de13fc admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:33.660283 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd15286-e6c1-44ee-8e75-0f72c3de13fc admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:33.665429 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4593c706-c998-4042-b5f5-a8bd743c3083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:33.666096 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593c706-c998-4042-b5f5-a8bd743c3083 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:33.666393 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593c706-c998-4042-b5f5-a8bd743c3083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:33.666666 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593c706-c998-4042-b5f5-a8bd743c3083 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:33.667451 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4593c706-c998-4042-b5f5-a8bd743c3083 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:33.669073 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4593c706-c998-4042-b5f5-a8bd743c3083 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:33.680784 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1419/2849] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 16:29:33.685305 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44606b4f-d3ca-4f0a-a1cf-c7620f16065d admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:33.695105 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44606b4f-d3ca-4f0a-a1cf-c7620f16065d admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:33.700041 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d60034c-7fee-4bda-b911-e0d0fa139423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:33.700930 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60034c-7fee-4bda-b911-e0d0fa139423 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:33.701219 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60034c-7fee-4bda-b911-e0d0fa139423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:33.701499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60034c-7fee-4bda-b911-e0d0fa139423 admin admin] PATH_INFO: `/v3/projects/501da3536d6c41c5b8838bd643c33b22/users/ed482cfcdfe4460fb158ca5d8c92fc99/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:33.716945 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d60034c-7fee-4bda-b911-e0d0fa139423 admin admin] RBAC: Authorizing `identity:create_grant(project_id=501da3536d6c41c5b8838bd643c33b22, user_id=ed482cfcdfe4460fb158ca5d8c92fc99, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:33.719634 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d60034c-7fee-4bda-b911-e0d0fa139423 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:33.729420 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:33.740053 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1431/2850] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:29:33 2022] PUT /identity/v3/projects/501da3536d6c41c5b8838bd643c33b22/users/ed482cfcdfe4460fb158ca5d8c92fc99/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:33.742180 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2befa-67cc-40cb-bda9-87ea02caa95d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:33.743194 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2befa-67cc-40cb-bda9-87ea02caa95d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:33.743464 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2befa-67cc-40cb-bda9-87ea02caa95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:33.772783 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-18c2befa-67cc-40cb-bda9-87ea02caa95d None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:29:33.774152 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-18c2befa-67cc-40cb-bda9-87ea02caa95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:33.840848 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1420/2851] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:29:33.854048 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d60034c-7fee-4bda-b911-e0d0fa139423 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:33.856536 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-7d60034c-7fee-4bda-b911-e0d0fa139423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:33.914878 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d60034c-7fee-4bda-b911-e0d0fa139423 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:33.918836 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d82a6-1dd3-48ab-8ec0-d65879917b1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:33.919516 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d82a6-1dd3-48ab-8ec0-d65879917b1f service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:33.919800 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d82a6-1dd3-48ab-8ec0-d65879917b1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:33.920075 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d82a6-1dd3-48ab-8ec0-d65879917b1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:33.929732 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72d82a6-1dd3-48ab-8ec0-d65879917b1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:33.931148 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72d82a6-1dd3-48ab-8ec0-d65879917b1f service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:33.940659 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1432/2852] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:29:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:29:34.177590 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e474d9-2080-4577-b7dd-492dbe72ac86 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:34.177891 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e474d9-2080-4577-b7dd-492dbe72ac86 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:34.178178 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e474d9-2080-4577-b7dd-492dbe72ac86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:34.209363 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-06e474d9-2080-4577-b7dd-492dbe72ac86 None None] MFA Rules not processed for user `865b66dd2b0f4ac8822ffaf219ad3a77`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:29:34.211070 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-06e474d9-2080-4577-b7dd-492dbe72ac86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:34.255027 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:34.306271 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1421/2853] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:29:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:29:34.311241 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d82a6-1dd3-48ab-8ec0-d65879917b1f service neutron] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:34.312819 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-d72d82a6-1dd3-48ab-8ec0-d65879917b1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:34.327067 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d82a6-1dd3-48ab-8ec0-d65879917b1f service neutron] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:34.332684 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c307fe-d1c1-42b0-ae8e-d7c224e2b692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:34.333372 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c307fe-d1c1-42b0-ae8e-d7c224e2b692 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:34.333703 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c307fe-d1c1-42b0-ae8e-d7c224e2b692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:34.333925 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c307fe-d1c1-42b0-ae8e-d7c224e2b692 admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:34.334629 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c307fe-d1c1-42b0-ae8e-d7c224e2b692 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:34.336638 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c307fe-d1c1-42b0-ae8e-d7c224e2b692 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:34.369780 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1433/2854] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:29:34 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:29:34.374357 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e474d9-2080-4577-b7dd-492dbe72ac86 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:34.387477 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e474d9-2080-4577-b7dd-492dbe72ac86 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:34.459651 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631d25c6-8bdc-4a1e-9c77-8c939014bc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:34.460486 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d25c6-8bdc-4a1e-9c77-8c939014bc7e admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:34.461040 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d25c6-8bdc-4a1e-9c77-8c939014bc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:34.461239 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d25c6-8bdc-4a1e-9c77-8c939014bc7e admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:34.462152 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631d25c6-8bdc-4a1e-9c77-8c939014bc7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:34.464569 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631d25c6-8bdc-4a1e-9c77-8c939014bc7e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:34.492910 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1422/2855] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:29:34 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:29:34.494442 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c307fe-d1c1-42b0-ae8e-d7c224e2b692 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:34.507515 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c307fe-d1c1-42b0-ae8e-d7c224e2b692 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:34.513071 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17ef6a-f12d-4f64-89fc-060013c10a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:34.513745 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17ef6a-f12d-4f64-89fc-060013c10a43 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:34.514032 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17ef6a-f12d-4f64-89fc-060013c10a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:34.514302 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17ef6a-f12d-4f64-89fc-060013c10a43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:34.515048 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c17ef6a-f12d-4f64-89fc-060013c10a43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:34.516836 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c17ef6a-f12d-4f64-89fc-060013c10a43 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:34.531411 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1434/2856] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:29:34.535441 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631d25c6-8bdc-4a1e-9c77-8c939014bc7e admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:34.551665 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631d25c6-8bdc-4a1e-9c77-8c939014bc7e admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:34.556425 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a2aa7f-54d3-44bd-b165-c743a54402af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:34.557323 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2aa7f-54d3-44bd-b165-c743a54402af admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:34.557620 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2aa7f-54d3-44bd-b165-c743a54402af admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:34.557894 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2aa7f-54d3-44bd-b165-c743a54402af admin admin] PATH_INFO: `/v3/projects/05799ea0163a4c09aa5201d401204487/users/539973a6c2524be0a616701f989c6653/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:34.574334 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a2aa7f-54d3-44bd-b165-c743a54402af admin admin] RBAC: Authorizing `identity:create_grant(project_id=05799ea0163a4c09aa5201d401204487, user_id=539973a6c2524be0a616701f989c6653, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:34.577100 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a2aa7f-54d3-44bd-b165-c743a54402af admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:34.591435 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1423/2857] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:29:34 2022] PUT /identity/v3/projects/05799ea0163a4c09aa5201d401204487/users/539973a6c2524be0a616701f989c6653/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:34.595227 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17ef6a-f12d-4f64-89fc-060013c10a43 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:34.606128 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17ef6a-f12d-4f64-89fc-060013c10a43 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:34.665954 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f614310a-9096-428e-aa09-9c57cbfaed5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:34.666715 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614310a-9096-428e-aa09-9c57cbfaed5c admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:34.667072 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614310a-9096-428e-aa09-9c57cbfaed5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:34.667388 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614310a-9096-428e-aa09-9c57cbfaed5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:34.668117 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f614310a-9096-428e-aa09-9c57cbfaed5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:34.669756 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f614310a-9096-428e-aa09-9c57cbfaed5c admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:34.681329 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1435/2858] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:29:34.685914 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a2aa7f-54d3-44bd-b165-c743a54402af admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:34.701457 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a2aa7f-54d3-44bd-b165-c743a54402af admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:34.711188 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b2ad4-d2b9-4b9f-bf8f-99c434c0eca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:34.712153 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b2ad4-d2b9-4b9f-bf8f-99c434c0eca0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:34.712440 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b2ad4-d2b9-4b9f-bf8f-99c434c0eca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:34.712722 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b2ad4-d2b9-4b9f-bf8f-99c434c0eca0 admin admin] PATH_INFO: `/v3/projects/05799ea0163a4c09aa5201d401204487/users/539973a6c2524be0a616701f989c6653/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:34.714803 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772b2ad4-d2b9-4b9f-bf8f-99c434c0eca0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05799ea0163a4c09aa5201d401204487, user_id=539973a6c2524be0a616701f989c6653, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:34.717463 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772b2ad4-d2b9-4b9f-bf8f-99c434c0eca0 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:34.729434 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1424/2859] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:29:34 2022] PUT /identity/v3/projects/05799ea0163a4c09aa5201d401204487/users/539973a6c2524be0a616701f989c6653/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:35.494789 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7258bd-98ed-4a02-8f76-cadf1ed693a6 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:35.496408 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7258bd-98ed-4a02-8f76-cadf1ed693a6 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:35.496857 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7258bd-98ed-4a02-8f76-cadf1ed693a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:35.532390 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-2d7258bd-98ed-4a02-8f76-cadf1ed693a6 None None] MFA Rules not processed for user `539973a6c2524be0a616701f989c6653`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:29:35.533648 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-2d7258bd-98ed-4a02-8f76-cadf1ed693a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:35.625832 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:35.656607 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1436/2860] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:29:35 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:29:35.668788 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b2ad4-d2b9-4b9f-bf8f-99c434c0eca0 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:35.670746 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-772b2ad4-d2b9-4b9f-bf8f-99c434c0eca0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:35.763598 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b2ad4-d2b9-4b9f-bf8f-99c434c0eca0 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:35.767356 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e042a-2af9-431d-975c-e0bb883f0694 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:35.768105 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e042a-2af9-431d-975c-e0bb883f0694 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:35.769061 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e042a-2af9-431d-975c-e0bb883f0694 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:35.769440 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e042a-2af9-431d-975c-e0bb883f0694 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:35.807896 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:35.840541 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102e042a-2af9-431d-975c-e0bb883f0694 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:35.841999 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102e042a-2af9-431d-975c-e0bb883f0694 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:35.852373 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1425/2861] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:29:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:29:37.229777 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7258bd-98ed-4a02-8f76-cadf1ed693a6 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:37.232886 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-2d7258bd-98ed-4a02-8f76-cadf1ed693a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:37.245929 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7258bd-98ed-4a02-8f76-cadf1ed693a6 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:37.255548 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd8dbd-89b5-4b69-b7d1-09715a83c04b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62a3bbb9720049b29eb37b0dd88bbad7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:37.256225 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd8dbd-89b5-4b69-b7d1-09715a83c04b service cinder] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:37.256514 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd8dbd-89b5-4b69-b7d1-09715a83c04b service cinder] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:37.256793 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd8dbd-89b5-4b69-b7d1-09715a83c04b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:37.267433 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cd8dbd-89b5-4b69-b7d1-09715a83c04b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:37.268929 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cd8dbd-89b5-4b69-b7d1-09715a83c04b service cinder] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:37.281030 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1437/2862] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:29:37 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:29:38.505943 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02bc455-db94-4ec6-b46f-280b141f73a7 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:38.506557 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02bc455-db94-4ec6-b46f-280b141f73a7 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:38.506557 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02bc455-db94-4ec6-b46f-280b141f73a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:38.533702 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-a02bc455-db94-4ec6-b46f-280b141f73a7 None None] MFA Rules not processed for user `ed482cfcdfe4460fb158ca5d8c92fc99`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:29:38.534898 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a02bc455-db94-4ec6-b46f-280b141f73a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:38.621088 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1426/2863] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:29:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:29:38.625600 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd8dbd-89b5-4b69-b7d1-09715a83c04b service cinder] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:38.693083 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd8dbd-89b5-4b69-b7d1-09715a83c04b service cinder] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:38.698093 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134038d8-e74c-4dbb-8fb7-062e7fc7e1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:38.698831 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134038d8-e74c-4dbb-8fb7-062e7fc7e1bd admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:38.699107 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134038d8-e74c-4dbb-8fb7-062e7fc7e1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:38.699412 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134038d8-e74c-4dbb-8fb7-062e7fc7e1bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:38.700111 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134038d8-e74c-4dbb-8fb7-062e7fc7e1bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:38.702009 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134038d8-e74c-4dbb-8fb7-062e7fc7e1bd admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:38.731645 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1438/2864] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:29:38 2022] POST /identity/v3/projects => generated 371 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:29:38.736026 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02bc455-db94-4ec6-b46f-280b141f73a7 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:38.748105 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02bc455-db94-4ec6-b46f-280b141f73a7 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:38.812192 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff1df99-6cc7-4f04-bf94-2305212565ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:38.812963 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff1df99-6cc7-4f04-bf94-2305212565ae admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:38.813320 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff1df99-6cc7-4f04-bf94-2305212565ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:38.813665 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff1df99-6cc7-4f04-bf94-2305212565ae admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:38.814454 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff1df99-6cc7-4f04-bf94-2305212565ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:38.817179 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff1df99-6cc7-4f04-bf94-2305212565ae admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:38.838730 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1427/2865] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:29:38 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:29:38.842168 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134038d8-e74c-4dbb-8fb7-062e7fc7e1bd admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:38.855313 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134038d8-e74c-4dbb-8fb7-062e7fc7e1bd admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:38.860406 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb92b655-4744-498a-8a10-6753a98ad09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:38.861309 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92b655-4744-498a-8a10-6753a98ad09c admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:38.861636 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92b655-4744-498a-8a10-6753a98ad09c admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:38.862818 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92b655-4744-498a-8a10-6753a98ad09c admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:38.863649 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb92b655-4744-498a-8a10-6753a98ad09c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:38.865250 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb92b655-4744-498a-8a10-6753a98ad09c admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:38.880038 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1439/2866] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:29:38.881557 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff1df99-6cc7-4f04-bf94-2305212565ae admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:38.893123 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff1df99-6cc7-4f04-bf94-2305212565ae admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:38.900289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a955995-db33-43d0-af51-544953968fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:38.901618 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a955995-db33-43d0-af51-544953968fd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:38.902004 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a955995-db33-43d0-af51-544953968fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:38.902393 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a955995-db33-43d0-af51-544953968fd8 admin admin] PATH_INFO: `/v3/projects/ac6dce4269204d12bb126054ca8f8442/users/25bdbb0019cd4fed8ce5fd37e585a43e/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:38.920898 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a955995-db33-43d0-af51-544953968fd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac6dce4269204d12bb126054ca8f8442, user_id=25bdbb0019cd4fed8ce5fd37e585a43e, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:38.923631 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a955995-db33-43d0-af51-544953968fd8 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:38.939995 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1428/2867] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:29:38 2022] PUT /identity/v3/projects/ac6dce4269204d12bb126054ca8f8442/users/25bdbb0019cd4fed8ce5fd37e585a43e/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:29:38.944587 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb92b655-4744-498a-8a10-6753a98ad09c admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:38.956744 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb92b655-4744-498a-8a10-6753a98ad09c admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:39.028611 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069179d-39d7-43a8-a63f-5135111f7b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:39.029333 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069179d-39d7-43a8-a63f-5135111f7b29 admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:39.029618 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069179d-39d7-43a8-a63f-5135111f7b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:39.030004 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069179d-39d7-43a8-a63f-5135111f7b29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:39.030751 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8069179d-39d7-43a8-a63f-5135111f7b29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:39.032596 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8069179d-39d7-43a8-a63f-5135111f7b29 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:39.050781 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1440/2868] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:29:39.052522 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a955995-db33-43d0-af51-544953968fd8 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:39.065976 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a955995-db33-43d0-af51-544953968fd8 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:39.073390 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb070eca-e2df-4b1c-aa04-9b5957c7ebbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:39.074466 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb070eca-e2df-4b1c-aa04-9b5957c7ebbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:39.074773 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb070eca-e2df-4b1c-aa04-9b5957c7ebbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:39.075057 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb070eca-e2df-4b1c-aa04-9b5957c7ebbd admin admin] PATH_INFO: `/v3/projects/ac6dce4269204d12bb126054ca8f8442/users/25bdbb0019cd4fed8ce5fd37e585a43e/roles/75936a608b8149099ff54e317f3a6740` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:39.100914 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb070eca-e2df-4b1c-aa04-9b5957c7ebbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac6dce4269204d12bb126054ca8f8442, user_id=25bdbb0019cd4fed8ce5fd37e585a43e, role_id=75936a608b8149099ff54e317f3a6740)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:39.105631 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb070eca-e2df-4b1c-aa04-9b5957c7ebbd admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:39.126621 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1429/2869] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:29:39 2022] PUT /identity/v3/projects/ac6dce4269204d12bb126054ca8f8442/users/25bdbb0019cd4fed8ce5fd37e585a43e/roles/75936a608b8149099ff54e317f3a6740 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:43.512580 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f14d72-f5b7-4fe1-a12f-f728f82e75e9 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:43.513044 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f14d72-f5b7-4fe1-a12f-f728f82e75e9 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:43.513242 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f14d72-f5b7-4fe1-a12f-f728f82e75e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:43.542944 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-c8f14d72-f5b7-4fe1-a12f-f728f82e75e9 None None] MFA Rules not processed for user `25bdbb0019cd4fed8ce5fd37e585a43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:29:43.544139 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-c8f14d72-f5b7-4fe1-a12f-f728f82e75e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:43.579837 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:43.619029 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1441/2870] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:29:43 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:29:43.634680 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb070eca-e2df-4b1c-aa04-9b5957c7ebbd admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:43.636585 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-eb070eca-e2df-4b1c-aa04-9b5957c7ebbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:43.653316 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:43.715655 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb070eca-e2df-4b1c-aa04-9b5957c7ebbd admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:43.722195 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8833c556-89d7-472d-a0c3-5a1dba3c26b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:43.722844 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833c556-89d7-472d-a0c3-5a1dba3c26b2 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:43.723125 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833c556-89d7-472d-a0c3-5a1dba3c26b2 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:43.723439 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833c556-89d7-472d-a0c3-5a1dba3c26b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:43.781205 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8833c556-89d7-472d-a0c3-5a1dba3c26b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:43.782954 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8833c556-89d7-472d-a0c3-5a1dba3c26b2 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:43.793953 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1430/2871] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:29:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:29:44.922833 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df58ef94-1e8e-4d6c-a149-f2a3f1cfab0e None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:44.923395 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df58ef94-1e8e-4d6c-a149-f2a3f1cfab0e None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:44.923533 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df58ef94-1e8e-4d6c-a149-f2a3f1cfab0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:44.942366 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-df58ef94-1e8e-4d6c-a149-f2a3f1cfab0e None None] MFA Rules not processed for user `ed482cfcdfe4460fb158ca5d8c92fc99`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:29:44.943656 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-df58ef94-1e8e-4d6c-a149-f2a3f1cfab0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:44.950698 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1442/2872] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:29:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:29:51.105920 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8833c556-89d7-472d-a0c3-5a1dba3c26b2 service nova] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:51.172289 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8833c556-89d7-472d-a0c3-5a1dba3c26b2 service nova] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:51.177298 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819ed72-a450-4c04-87c6-60c651d5c27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:51.178153 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819ed72-a450-4c04-87c6-60c651d5c27f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:51.178434 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819ed72-a450-4c04-87c6-60c651d5c27f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:51.178876 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819ed72-a450-4c04-87c6-60c651d5c27f admin admin] PATH_INFO: `/v3/users/865b66dd2b0f4ac8822ffaf219ad3a77` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:51.180083 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1819ed72-a450-4c04-87c6-60c651d5c27f admin admin] RBAC: Authorizing `identity:delete_user(user_id=865b66dd2b0f4ac8822ffaf219ad3a77)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:51.181996 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1819ed72-a450-4c04-87c6-60c651d5c27f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:51.228407 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1819ed72-a450-4c04-87c6-60c651d5c27f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '865b66dd2b0f4ac8822ffaf219ad3a77'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:51.229573 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:51.234223 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1819ed72-a450-4c04-87c6-60c651d5c27f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '865b66dd2b0f4ac8822ffaf219ad3a77'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:51.247795 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1819ed72-a450-4c04-87c6-60c651d5c27f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '865b66dd2b0f4ac8822ffaf219ad3a77'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:51.252106 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1819ed72-a450-4c04-87c6-60c651d5c27f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '865b66dd2b0f4ac8822ffaf219ad3a77'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:51.259919 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1819ed72-a450-4c04-87c6-60c651d5c27f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '865b66dd2b0f4ac8822ffaf219ad3a77'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:51.265885 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1431/2873] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:29:51 2022] DELETE /identity/v3/users/865b66dd2b0f4ac8822ffaf219ad3a77 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:51.441264 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df58ef94-1e8e-4d6c-a149-f2a3f1cfab0e None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:51.452433 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df58ef94-1e8e-4d6c-a149-f2a3f1cfab0e None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:51.508489 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b676310b-defd-4fba-899d-c1bd955c2575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:51.509282 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676310b-defd-4fba-899d-c1bd955c2575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:51.509562 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676310b-defd-4fba-899d-c1bd955c2575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:51.509836 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676310b-defd-4fba-899d-c1bd955c2575 admin admin] PATH_INFO: `/v3/projects/896eaf64068542099d21437fb023e734` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:51.510996 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b676310b-defd-4fba-899d-c1bd955c2575 admin admin] RBAC: Authorizing `identity:delete_project(project_id=896eaf64068542099d21437fb023e734)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:51.512908 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b676310b-defd-4fba-899d-c1bd955c2575 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:51.557979 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-b676310b-defd-4fba-899d-c1bd955c2575 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '896eaf64068542099d21437fb023e734', 'request_id': 'req-b676310b-defd-4fba-899d-c1bd955c2575'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:51.571318 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-b676310b-defd-4fba-899d-c1bd955c2575 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '896eaf64068542099d21437fb023e734', 'request_id': 'req-b676310b-defd-4fba-899d-c1bd955c2575'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:51.572829 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-b676310b-defd-4fba-899d-c1bd955c2575 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '896eaf64068542099d21437fb023e734', 'request_id': 'req-b676310b-defd-4fba-899d-c1bd955c2575'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:51.584689 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-b676310b-defd-4fba-899d-c1bd955c2575 admin admin] The token cache is being invalidate because project 896eaf64068542099d21437fb023e734 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:29:51.584966 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-b676310b-defd-4fba-899d-c1bd955c2575 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:51.585668 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-b676310b-defd-4fba-899d-c1bd955c2575 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:51.590549 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1443/2874] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:29:51 2022] DELETE /identity/v3/projects/896eaf64068542099d21437fb023e734 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:51.591851 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819ed72-a450-4c04-87c6-60c651d5c27f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:51.593383 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1819ed72-a450-4c04-87c6-60c651d5c27f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:51.607462 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819ed72-a450-4c04-87c6-60c651d5c27f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:51.667012 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808eb11-2cbb-4ce9-98d6-d22786fff57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:51.667945 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808eb11-2cbb-4ce9-98d6-d22786fff57a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:51.668210 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808eb11-2cbb-4ce9-98d6-d22786fff57a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:51.668484 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808eb11-2cbb-4ce9-98d6-d22786fff57a admin admin] PATH_INFO: `/v3/users/539973a6c2524be0a616701f989c6653` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:51.669674 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0808eb11-2cbb-4ce9-98d6-d22786fff57a admin admin] RBAC: Authorizing `identity:delete_user(user_id=539973a6c2524be0a616701f989c6653)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:51.671623 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0808eb11-2cbb-4ce9-98d6-d22786fff57a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:51.723417 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0808eb11-2cbb-4ce9-98d6-d22786fff57a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '539973a6c2524be0a616701f989c6653'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:51.730924 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0808eb11-2cbb-4ce9-98d6-d22786fff57a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '539973a6c2524be0a616701f989c6653'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:51.742652 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0808eb11-2cbb-4ce9-98d6-d22786fff57a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '539973a6c2524be0a616701f989c6653'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:51.743262 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0808eb11-2cbb-4ce9-98d6-d22786fff57a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '539973a6c2524be0a616701f989c6653'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:51.750039 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0808eb11-2cbb-4ce9-98d6-d22786fff57a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '539973a6c2524be0a616701f989c6653'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:51.755992 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1432/2875] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:29:51 2022] DELETE /identity/v3/users/539973a6c2524be0a616701f989c6653 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:51.934915 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b676310b-defd-4fba-899d-c1bd955c2575 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:51.936536 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-b676310b-defd-4fba-899d-c1bd955c2575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:51.949548 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b676310b-defd-4fba-899d-c1bd955c2575 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:52.002123 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:52.010713 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47435e9c-97fc-428f-8630-e1791728293e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:52.011740 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47435e9c-97fc-428f-8630-e1791728293e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:52.012090 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47435e9c-97fc-428f-8630-e1791728293e admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:52.012548 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47435e9c-97fc-428f-8630-e1791728293e admin admin] PATH_INFO: `/v3/projects/05799ea0163a4c09aa5201d401204487` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:52.013904 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47435e9c-97fc-428f-8630-e1791728293e admin admin] RBAC: Authorizing `identity:delete_project(project_id=05799ea0163a4c09aa5201d401204487)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:52.015923 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47435e9c-97fc-428f-8630-e1791728293e admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:52.061979 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-47435e9c-97fc-428f-8630-e1791728293e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05799ea0163a4c09aa5201d401204487', 'request_id': 'req-47435e9c-97fc-428f-8630-e1791728293e'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:52.069626 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-47435e9c-97fc-428f-8630-e1791728293e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05799ea0163a4c09aa5201d401204487', 'request_id': 'req-47435e9c-97fc-428f-8630-e1791728293e'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:52.080518 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-47435e9c-97fc-428f-8630-e1791728293e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05799ea0163a4c09aa5201d401204487', 'request_id': 'req-47435e9c-97fc-428f-8630-e1791728293e'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:52.095263 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-47435e9c-97fc-428f-8630-e1791728293e admin admin] The token cache is being invalidate because project 05799ea0163a4c09aa5201d401204487 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80286) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:29:52.095558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-47435e9c-97fc-428f-8630-e1791728293e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:52.096219 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-47435e9c-97fc-428f-8630-e1791728293e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:29:52.097789 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1444/2876] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:29:51 2022] DELETE /identity/v3/projects/05799ea0163a4c09aa5201d401204487 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:29:52.102714 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f05d3-c59f-4ab6-b014-eeb7860f907a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:52.103036 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f05d3-c59f-4ab6-b014-eeb7860f907a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:52.103341 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f05d3-c59f-4ab6-b014-eeb7860f907a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:52.120425 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-125f05d3-c59f-4ab6-b014-eeb7860f907a None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:29:52.121618 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-125f05d3-c59f-4ab6-b014-eeb7860f907a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:52.166904 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:29:52.182515 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1433/2877] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:29:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:29:52.190568 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c0661-39fc-4ed1-90cc-41bd3caf7dd1 None None] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:52.191070 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c0661-39fc-4ed1-90cc-41bd3caf7dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:52.191385 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c0661-39fc-4ed1-90cc-41bd3caf7dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:52.210398 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.auth.core [None req-827c0661-39fc-4ed1-90cc-41bd3caf7dd1 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80286) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:29:52.211733 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-827c0661-39fc-4ed1-90cc-41bd3caf7dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:52.219578 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1445/2878] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:29:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:29:52.223732 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125f05d3-c59f-4ab6-b014-eeb7860f907a None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:52.233268 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125f05d3-c59f-4ab6-b014-eeb7860f907a None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:52.237794 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f62985-c3a4-4568-a6a8-a85e7038b14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:52.238478 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f62985-c3a4-4568-a6a8-a85e7038b14e admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:52.238756 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f62985-c3a4-4568-a6a8-a85e7038b14e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:52.239028 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f62985-c3a4-4568-a6a8-a85e7038b14e admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:52.239784 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f62985-c3a4-4568-a6a8-a85e7038b14e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 16:29:52.240088 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f62985-c3a4-4568-a6a8-a85e7038b14e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:52.241665 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f62985-c3a4-4568-a6a8-a85e7038b14e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:52.253931 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1434/2879] 10.222.0.41 () {58 vars in 1152 bytes} [Fri Jun 3 16:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 16:29:52.258198 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c0661-39fc-4ed1-90cc-41bd3caf7dd1 None None] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:52.267551 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c0661-39fc-4ed1-90cc-41bd3caf7dd1 None None] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:52.272842 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fff79bb-b902-47d6-82f4-aa2f54242ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:52.273489 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff79bb-b902-47d6-82f4-aa2f54242ea9 admin admin] REQUEST_METHOD: `POST` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:52.273777 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff79bb-b902-47d6-82f4-aa2f54242ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:52.274060 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff79bb-b902-47d6-82f4-aa2f54242ea9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:52.274757 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fff79bb-b902-47d6-82f4-aa2f54242ea9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:52.276681 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fff79bb-b902-47d6-82f4-aa2f54242ea9 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:52.301743 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1446/2880] 10.222.0.41 () {60 vars in 1152 bytes} [Fri Jun 3 16:29:52 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:29:52.305815 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f62985-c3a4-4568-a6a8-a85e7038b14e admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:52.316540 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f62985-c3a4-4568-a6a8-a85e7038b14e admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:52.378898 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751430d-be88-4df7-a90e-78025ba40ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:52.379628 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751430d-be88-4df7-a90e-78025ba40ebe admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:52.379915 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751430d-be88-4df7-a90e-78025ba40ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:52.380196 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751430d-be88-4df7-a90e-78025ba40ebe admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:52.380872 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f751430d-be88-4df7-a90e-78025ba40ebe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:52.382815 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f751430d-be88-4df7-a90e-78025ba40ebe admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:52.405516 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1435/2881] 10.222.0.41 () {60 vars in 1143 bytes} [Fri Jun 3 16:29:52 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 16:29:52.406902 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fff79bb-b902-47d6-82f4-aa2f54242ea9 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:52.416930 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fff79bb-b902-47d6-82f4-aa2f54242ea9 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:52.421437 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee26ae4-7f59-4a6b-a85d-f2f3387d5a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:52.422094 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee26ae4-7f59-4a6b-a85d-f2f3387d5a8a admin admin] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:52.422380 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee26ae4-7f59-4a6b-a85d-f2f3387d5a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:52.422658 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee26ae4-7f59-4a6b-a85d-f2f3387d5a8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:52.423418 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee26ae4-7f59-4a6b-a85d-f2f3387d5a8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:52.425497 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee26ae4-7f59-4a6b-a85d-f2f3387d5a8a admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:52.435702 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1447/2882] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:29:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:29:52.439513 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751430d-be88-4df7-a90e-78025ba40ebe admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:52.449880 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751430d-be88-4df7-a90e-78025ba40ebe admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:52.454754 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772502cb-1be0-4319-80a8-6a62f60356d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:52.455456 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772502cb-1be0-4319-80a8-6a62f60356d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:52.455774 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772502cb-1be0-4319-80a8-6a62f60356d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:52.456079 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772502cb-1be0-4319-80a8-6a62f60356d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:52.456826 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772502cb-1be0-4319-80a8-6a62f60356d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:52.458459 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772502cb-1be0-4319-80a8-6a62f60356d8 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:52.468467 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1436/2883] 10.222.0.41 () {58 vars in 1121 bytes} [Fri Jun 3 16:29:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 16:29:52.472213 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee26ae4-7f59-4a6b-a85d-f2f3387d5a8a admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:52.481989 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee26ae4-7f59-4a6b-a85d-f2f3387d5a8a admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:52.486777 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6882fe0f-797b-4322-b115-82b5af130c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:52.488247 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882fe0f-797b-4322-b115-82b5af130c63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:52.488533 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882fe0f-797b-4322-b115-82b5af130c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:52.488810 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882fe0f-797b-4322-b115-82b5af130c63 admin admin] PATH_INFO: `/v3/projects/58f08c6ab27b4261bca907935639f2b7/users/94c352b8314b4b399ce4db2796afd639/roles/a8e4174e3572497f9284c709a119dcff` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:52.503939 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6882fe0f-797b-4322-b115-82b5af130c63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58f08c6ab27b4261bca907935639f2b7, user_id=94c352b8314b4b399ce4db2796afd639, role_id=a8e4174e3572497f9284c709a119dcff)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:52.506570 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6882fe0f-797b-4322-b115-82b5af130c63 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:52.519513 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1448/2884] 10.222.0.41 () {60 vars in 1482 bytes} [Fri Jun 3 16:29:52 2022] PUT /identity/v3/projects/58f08c6ab27b4261bca907935639f2b7/users/94c352b8314b4b399ce4db2796afd639/roles/a8e4174e3572497f9284c709a119dcff => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:29:52.529798 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f085b-b808-4e8f-9968-7c6076b9b901 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:52.530098 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f085b-b808-4e8f-9968-7c6076b9b901 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:52.530384 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f085b-b808-4e8f-9968-7c6076b9b901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:52.557093 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-1f4f085b-b808-4e8f-9968-7c6076b9b901 None None] MFA Rules not processed for user `94c352b8314b4b399ce4db2796afd639`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:29:52.558347 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1f4f085b-b808-4e8f-9968-7c6076b9b901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:52.629980 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1437/2885] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:29:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 16:29:52.641335 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6882fe0f-797b-4322-b115-82b5af130c63 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:29:52.643082 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-6882fe0f-797b-4322-b115-82b5af130c63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:29:52.699063 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6882fe0f-797b-4322-b115-82b5af130c63 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:29:52.702818 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19324ed3-9686-4ee2-b004-25c0b483d7af service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbaf66d6e1f54047917cc615546bbd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:29:52.703500 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19324ed3-9686-4ee2-b004-25c0b483d7af service glance] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:29:52.703794 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19324ed3-9686-4ee2-b004-25c0b483d7af service glance] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:29:52.704073 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19324ed3-9686-4ee2-b004-25c0b483d7af service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:29:52.712388 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19324ed3-9686-4ee2-b004-25c0b483d7af service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:29:52.713813 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19324ed3-9686-4ee2-b004-25c0b483d7af service glance] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:29:52.722803 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1449/2886] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:29:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:30:02.626348 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=77,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:30:02.718689 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:30:11.962365 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4f085b-b808-4e8f-9968-7c6076b9b901 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:30:11.963836 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1f4f085b-b808-4e8f-9968-7c6076b9b901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:30:12.022380 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4f085b-b808-4e8f-9968-7c6076b9b901 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:30:12.026634 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b65ba9-6c20-479d-8ca9-a1b280b47853 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:30:12.027301 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b65ba9-6c20-479d-8ca9-a1b280b47853 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:30:12.027610 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b65ba9-6c20-479d-8ca9-a1b280b47853 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:30:12.027887 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b65ba9-6c20-479d-8ca9-a1b280b47853 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:30:12.030076 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d6b65ba9-6c20-479d-8ca9-a1b280b47853 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:30:12.045720 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b65ba9-6c20-479d-8ca9-a1b280b47853 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:30:12.047189 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b65ba9-6c20-479d-8ca9-a1b280b47853 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:30:12.055918 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1438/2887] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:30:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:30:22.052283 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=26 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:30:36.131959 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19324ed3-9686-4ee2-b004-25c0b483d7af service glance] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:30:36.197218 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19324ed3-9686-4ee2-b004-25c0b483d7af service glance] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:30:36.201897 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ccbf81-2d7b-4d19-894c-c4200a4c7114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:30:36.202779 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ccbf81-2d7b-4d19-894c-c4200a4c7114 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:30:36.203066 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ccbf81-2d7b-4d19-894c-c4200a4c7114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:30:36.203380 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ccbf81-2d7b-4d19-894c-c4200a4c7114 admin admin] PATH_INFO: `/v3/users/94c352b8314b4b399ce4db2796afd639` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:30:36.204620 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ccbf81-2d7b-4d19-894c-c4200a4c7114 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94c352b8314b4b399ce4db2796afd639)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:30:36.206585 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ccbf81-2d7b-4d19-894c-c4200a4c7114 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:30:36.255277 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-86ccbf81-2d7b-4d19-894c-c4200a4c7114 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94c352b8314b4b399ce4db2796afd639'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:30:36.264392 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-86ccbf81-2d7b-4d19-894c-c4200a4c7114 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94c352b8314b4b399ce4db2796afd639'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:30:36.277898 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-86ccbf81-2d7b-4d19-894c-c4200a4c7114 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94c352b8314b4b399ce4db2796afd639'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:30:36.285110 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-86ccbf81-2d7b-4d19-894c-c4200a4c7114 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94c352b8314b4b399ce4db2796afd639'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:30:36.285840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-86ccbf81-2d7b-4d19-894c-c4200a4c7114 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94c352b8314b4b399ce4db2796afd639'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:30:36.291797 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1450/2888] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:30:36 2022] DELETE /identity/v3/users/94c352b8314b4b399ce4db2796afd639 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:30:36.296325 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94970f-0bd0-4e8a-9906-8683ca07c405 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:30:36.296856 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94970f-0bd0-4e8a-9906-8683ca07c405 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:30:36.296971 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94970f-0bd0-4e8a-9906-8683ca07c405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:30:36.315769 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-8c94970f-0bd0-4e8a-9906-8683ca07c405 None None] MFA Rules not processed for user `5aca81affd5542cf9480c6e76c8c5478`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 16:30:36.317679 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8c94970f-0bd0-4e8a-9906-8683ca07c405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:30:36.376014 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1439/2889] 10.222.0.41 () {58 vars in 957 bytes} [Fri Jun 3 16:30:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 16:30:36.386341 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ccbf81-2d7b-4d19-894c-c4200a4c7114 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:30:36.387931 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-86ccbf81-2d7b-4d19-894c-c4200a4c7114 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:30:36.444384 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ccbf81-2d7b-4d19-894c-c4200a4c7114 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:30:36.448053 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcf4e6-0d83-4e2b-ada0-bb5134d6504a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:30:36.448691 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcf4e6-0d83-4e2b-ada0-bb5134d6504a service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:30:36.448981 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcf4e6-0d83-4e2b-ada0-bb5134d6504a service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:30:36.449265 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcf4e6-0d83-4e2b-ada0-bb5134d6504a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:30:36.458193 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fcf4e6-0d83-4e2b-ada0-bb5134d6504a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:30:36.459656 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fcf4e6-0d83-4e2b-ada0-bb5134d6504a service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:30:36.469084 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1451/2890] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:30:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:30:36.740413 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94970f-0bd0-4e8a-9906-8683ca07c405 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:30:36.741900 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8c94970f-0bd0-4e8a-9906-8683ca07c405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:30:36.754513 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94970f-0bd0-4e8a-9906-8683ca07c405 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:30:36.759195 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673bd0a1-c593-4336-8c15-0df65d1198f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:30:36.760088 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673bd0a1-c593-4336-8c15-0df65d1198f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:30:36.760373 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673bd0a1-c593-4336-8c15-0df65d1198f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:30:36.760647 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673bd0a1-c593-4336-8c15-0df65d1198f1 admin admin] PATH_INFO: `/v3/projects/58f08c6ab27b4261bca907935639f2b7` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:30:36.761918 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673bd0a1-c593-4336-8c15-0df65d1198f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58f08c6ab27b4261bca907935639f2b7)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:30:36.764039 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673bd0a1-c593-4336-8c15-0df65d1198f1 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:30:36.813332 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-673bd0a1-c593-4336-8c15-0df65d1198f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58f08c6ab27b4261bca907935639f2b7', 'request_id': 'req-673bd0a1-c593-4336-8c15-0df65d1198f1'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:30:36.819518 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-673bd0a1-c593-4336-8c15-0df65d1198f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58f08c6ab27b4261bca907935639f2b7', 'request_id': 'req-673bd0a1-c593-4336-8c15-0df65d1198f1'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:30:36.826695 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-673bd0a1-c593-4336-8c15-0df65d1198f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58f08c6ab27b4261bca907935639f2b7', 'request_id': 'req-673bd0a1-c593-4336-8c15-0df65d1198f1'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:30:36.836781 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-673bd0a1-c593-4336-8c15-0df65d1198f1 admin admin] The token cache is being invalidate because project 58f08c6ab27b4261bca907935639f2b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:30:36.837073 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-673bd0a1-c593-4336-8c15-0df65d1198f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:30:36.837705 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-673bd0a1-c593-4336-8c15-0df65d1198f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:30:36.839252 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1440/2891] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:30:36 2022] DELETE /identity/v3/projects/58f08c6ab27b4261bca907935639f2b7 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:30:38.834031 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcf4e6-0d83-4e2b-ada0-bb5134d6504a service neutron] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:30:38.842531 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcf4e6-0d83-4e2b-ada0-bb5134d6504a service neutron] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:30:38.859857 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:30:38.893047 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3400ef-b8d8-4196-b1f3-9448a6e6cedc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:30:38.893761 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3400ef-b8d8-4196-b1f3-9448a6e6cedc service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:30:38.894120 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3400ef-b8d8-4196-b1f3-9448a6e6cedc service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:30:38.894493 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3400ef-b8d8-4196-b1f3-9448a6e6cedc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:30:38.896276 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-3f3400ef-b8d8-4196-b1f3-9448a6e6cedc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:30:38.956299 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3400ef-b8d8-4196-b1f3-9448a6e6cedc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:30:38.957752 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3400ef-b8d8-4196-b1f3-9448a6e6cedc service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:30:38.967142 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1452/2892] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:30:38 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:30:46.833932 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:30:48.962920 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=40 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:32:01.211112 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673bd0a1-c593-4336-8c15-0df65d1198f1 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:32:01.212642 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-673bd0a1-c593-4336-8c15-0df65d1198f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:32:01.225765 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673bd0a1-c593-4336-8c15-0df65d1198f1 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:32:01.229451 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9be5c8-6983-4510-a022-94c77f10c731 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:32:01.230096 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9be5c8-6983-4510-a022-94c77f10c731 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:32:01.230383 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9be5c8-6983-4510-a022-94c77f10c731 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:32:01.230662 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9be5c8-6983-4510-a022-94c77f10c731 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:32:01.232443 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1b9be5c8-6983-4510-a022-94c77f10c731 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:32:01.241933 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9be5c8-6983-4510-a022-94c77f10c731 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:32:01.243387 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9be5c8-6983-4510-a022-94c77f10c731 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:32:01.251522 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1441/2893] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:32:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:32:11.247729 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=6 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:32:54.802239 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3400ef-b8d8-4196-b1f3-9448a6e6cedc service neutron] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:32:54.803831 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-3f3400ef-b8d8-4196-b1f3-9448a6e6cedc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:32:54.815193 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3400ef-b8d8-4196-b1f3-9448a6e6cedc service neutron] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:32:54.818746 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd1346-0a81-41fd-916e-d7a5f433cff5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:32:54.819425 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd1346-0a81-41fd-916e-d7a5f433cff5 service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:32:54.819714 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd1346-0a81-41fd-916e-d7a5f433cff5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:32:54.819993 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd1346-0a81-41fd-916e-d7a5f433cff5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:32:54.821710 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-6bcd1346-0a81-41fd-916e-d7a5f433cff5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:32:54.893464 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd1346-0a81-41fd-916e-d7a5f433cff5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:32:54.894897 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd1346-0a81-41fd-916e-d7a5f433cff5 service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:32:54.906508 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1453/2894] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:32:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:32:56.081804 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9be5c8-6983-4510-a022-94c77f10c731 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:32:56.090234 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9be5c8-6983-4510-a022-94c77f10c731 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:32:56.094614 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34097fd2-99a2-41e0-a739-21274d30e82c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:32:56.095240 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34097fd2-99a2-41e0-a739-21274d30e82c service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:32:56.095552 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34097fd2-99a2-41e0-a739-21274d30e82c service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:32:56.095830 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34097fd2-99a2-41e0-a739-21274d30e82c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:32:56.097577 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-34097fd2-99a2-41e0-a739-21274d30e82c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:32:56.110431 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34097fd2-99a2-41e0-a739-21274d30e82c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:32:56.111957 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34097fd2-99a2-41e0-a739-21274d30e82c service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:32:56.122581 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1442/2895] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:32:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:33:04.900914 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=34 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:33:06.117770 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=6 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:33:12.765447 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd1346-0a81-41fd-916e-d7a5f433cff5 service neutron] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:33:12.774701 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd1346-0a81-41fd-916e-d7a5f433cff5 service neutron] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:33:12.778376 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c2fc5-1802-4435-982b-235e86cacbe2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:33:12.779019 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c2fc5-1802-4435-982b-235e86cacbe2 service neutron] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:33:12.779326 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c2fc5-1802-4435-982b-235e86cacbe2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:33:12.779618 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c2fc5-1802-4435-982b-235e86cacbe2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:33:12.781637 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-9b5c2fc5-1802-4435-982b-235e86cacbe2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:33:12.791839 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5c2fc5-1802-4435-982b-235e86cacbe2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:33:12.793268 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5c2fc5-1802-4435-982b-235e86cacbe2 service neutron] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:33:12.801897 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1454/2896] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:33:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:33:12.915551 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34097fd2-99a2-41e0-a739-21274d30e82c service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:33:12.924309 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34097fd2-99a2-41e0-a739-21274d30e82c service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:33:12.928066 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51450ca8-50f4-4f5a-a5cd-978046b3ce03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:33:12.928777 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51450ca8-50f4-4f5a-a5cd-978046b3ce03 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:33:12.929147 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51450ca8-50f4-4f5a-a5cd-978046b3ce03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:33:12.929549 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51450ca8-50f4-4f5a-a5cd-978046b3ce03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:33:12.931599 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-51450ca8-50f4-4f5a-a5cd-978046b3ce03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:33:12.945376 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51450ca8-50f4-4f5a-a5cd-978046b3ce03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:33:12.946791 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51450ca8-50f4-4f5a-a5cd-978046b3ce03 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:33:12.958088 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1443/2897] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:33:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:33:13.020377 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c2fc5-1802-4435-982b-235e86cacbe2 service neutron] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:33:13.021802 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-9b5c2fc5-1802-4435-982b-235e86cacbe2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:33:13.091240 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c2fc5-1802-4435-982b-235e86cacbe2 service neutron] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:33:13.096409 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06894f81-dfb7-425f-bece-df8aec922dfe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be901786f8934d559e0bac5d622f97cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:33:13.097658 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06894f81-dfb7-425f-bece-df8aec922dfe service placement] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:33:13.097949 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06894f81-dfb7-425f-bece-df8aec922dfe service placement] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:33:13.098228 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06894f81-dfb7-425f-bece-df8aec922dfe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:33:13.100110 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-06894f81-dfb7-425f-bece-df8aec922dfe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:33:13.111496 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06894f81-dfb7-425f-bece-df8aec922dfe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:33:13.113016 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06894f81-dfb7-425f-bece-df8aec922dfe service placement] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:33:13.122796 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1455/2898] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:33:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:33:14.351428 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51450ca8-50f4-4f5a-a5cd-978046b3ce03 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:33:14.352955 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-51450ca8-50f4-4f5a-a5cd-978046b3ce03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:33:14.371271 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51450ca8-50f4-4f5a-a5cd-978046b3ce03 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:33:14.377371 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fa025c-7662-45da-af00-473d2bdd1441 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:33:14.378147 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa025c-7662-45da-af00-473d2bdd1441 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:33:14.378464 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa025c-7662-45da-af00-473d2bdd1441 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:33:14.378924 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa025c-7662-45da-af00-473d2bdd1441 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:33:14.381207 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-41fa025c-7662-45da-af00-473d2bdd1441 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:33:14.398943 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fa025c-7662-45da-af00-473d2bdd1441 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:33:14.401466 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fa025c-7662-45da-af00-473d2bdd1441 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:33:14.416673 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1444/2899] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:33:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 16:33:23.118645 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=38 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:33:24.410300 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=12 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:33:59.934921 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06894f81-dfb7-425f-bece-df8aec922dfe service placement] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:33:59.945870 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06894f81-dfb7-425f-bece-df8aec922dfe service placement] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:33:59.950886 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d88523-7d3c-4edb-974d-292d733c55ac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be901786f8934d559e0bac5d622f97cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:33:59.951558 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d88523-7d3c-4edb-974d-292d733c55ac service placement] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:33:59.951848 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d88523-7d3c-4edb-974d-292d733c55ac service placement] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:33:59.952125 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d88523-7d3c-4edb-974d-292d733c55ac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:33:59.953883 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-11d88523-7d3c-4edb-974d-292d733c55ac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:33:59.966172 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d88523-7d3c-4edb-974d-292d733c55ac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:33:59.967773 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d88523-7d3c-4edb-974d-292d733c55ac service placement] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:33:59.977721 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1456/2900] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:33:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:34:09.973377 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=6 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:34:52.237438 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fa025c-7662-45da-af00-473d2bdd1441 service nova] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:34:52.246762 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fa025c-7662-45da-af00-473d2bdd1441 service nova] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:34:52.250326 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4878d-0d4b-433a-b021-2ed578a4263d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:34:52.250960 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4878d-0d4b-433a-b021-2ed578a4263d service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:34:52.251246 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4878d-0d4b-433a-b021-2ed578a4263d service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:34:52.251566 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4878d-0d4b-433a-b021-2ed578a4263d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:34:52.253345 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ced4878d-0d4b-433a-b021-2ed578a4263d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:34:52.314867 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced4878d-0d4b-433a-b021-2ed578a4263d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:34:52.316299 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced4878d-0d4b-433a-b021-2ed578a4263d service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:34:52.325432 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1445/2901] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:34:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:34:53.133318 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d88523-7d3c-4edb-974d-292d733c55ac service placement] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:34:53.134819 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-11d88523-7d3c-4edb-974d-292d733c55ac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:34:53.150840 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d88523-7d3c-4edb-974d-292d733c55ac service placement] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:34:53.158340 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d34a7-3b8f-4a73-ba7a-1438e539fa48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fd08ea39ee94b92ac6763608e69f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:34:53.159241 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d34a7-3b8f-4a73-ba7a-1438e539fa48 service nova] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:34:53.159704 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d34a7-3b8f-4a73-ba7a-1438e539fa48 service nova] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:34:53.160115 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d34a7-3b8f-4a73-ba7a-1438e539fa48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:34:53.162273 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-0c8d34a7-3b8f-4a73-ba7a-1438e539fa48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:34:53.178455 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8d34a7-3b8f-4a73-ba7a-1438e539fa48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:34:53.180756 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8d34a7-3b8f-4a73-ba7a-1438e539fa48 service nova] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:34:53.195047 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1457/2902] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:34:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:34:58.158401 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4878d-0d4b-433a-b021-2ed578a4263d service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:34:58.167557 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4878d-0d4b-433a-b021-2ed578a4263d service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:34:58.171836 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e1d30f-8ab6-4592-b1d7-1392f5a820cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aeeaad0680074179b3d29abcf0d51515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:34:58.172501 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1d30f-8ab6-4592-b1d7-1392f5a820cc service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:34:58.172788 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1d30f-8ab6-4592-b1d7-1392f5a820cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:34:58.173048 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1d30f-8ab6-4592-b1d7-1392f5a820cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:34:58.174855 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-00e1d30f-8ab6-4592-b1d7-1392f5a820cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:34:58.243305 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e1d30f-8ab6-4592-b1d7-1392f5a820cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:34:58.244760 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e1d30f-8ab6-4592-b1d7-1392f5a820cc service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:34:58.256730 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1446/2903] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:34:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 16:35:01.255465 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d34a7-3b8f-4a73-ba7a-1438e539fa48 service nova] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:35:01.257087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-0c8d34a7-3b8f-4a73-ba7a-1438e539fa48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:35:01.274579 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d34a7-3b8f-4a73-ba7a-1438e539fa48 service nova] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:35:01.286423 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d1bbaf-1626-44d5-b9ce-cbc1c54b0daa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be901786f8934d559e0bac5d622f97cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:35:01.288116 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1bbaf-1626-44d5-b9ce-cbc1c54b0daa service placement] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:35:01.288783 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1bbaf-1626-44d5-b9ce-cbc1c54b0daa service placement] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:35:01.289265 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1bbaf-1626-44d5-b9ce-cbc1c54b0daa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:35:01.293212 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-75d1bbaf-1626-44d5-b9ce-cbc1c54b0daa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:35:01.309087 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d1bbaf-1626-44d5-b9ce-cbc1c54b0daa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:35:01.311071 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d1bbaf-1626-44d5-b9ce-cbc1c54b0daa service placement] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:35:01.330861 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1458/2904] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:35:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:35:04.376574 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e1d30f-8ab6-4592-b1d7-1392f5a820cc service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:35:04.394535 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e1d30f-8ab6-4592-b1d7-1392f5a820cc service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:35:04.402763 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659a5bc-7d55-4d1b-beac-b8344a863c3f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be901786f8934d559e0bac5d622f97cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:35:04.403477 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659a5bc-7d55-4d1b-beac-b8344a863c3f service placement] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:35:04.403760 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659a5bc-7d55-4d1b-beac-b8344a863c3f service placement] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:35:04.404475 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659a5bc-7d55-4d1b-beac-b8344a863c3f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:35:04.408422 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6659a5bc-7d55-4d1b-beac-b8344a863c3f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:35:04.424794 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6659a5bc-7d55-4d1b-beac-b8344a863c3f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:35:04.428836 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d1bbaf-1626-44d5-b9ce-cbc1c54b0daa service placement] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:35:04.430508 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-75d1bbaf-1626-44d5-b9ce-cbc1c54b0daa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:35:04.436508 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6659a5bc-7d55-4d1b-beac-b8344a863c3f service placement] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:35:04.454665 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d1bbaf-1626-44d5-b9ce-cbc1c54b0daa service placement] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:35:04.460083 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf74f6-507b-4d0d-b74d-d44ceeaa5e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:35:04.461023 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf74f6-507b-4d0d-b74d-d44ceeaa5e0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:35:04.461378 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf74f6-507b-4d0d-b74d-d44ceeaa5e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:35:04.461728 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf74f6-507b-4d0d-b74d-d44ceeaa5e0a admin admin] PATH_INFO: `/v3/users/ed482cfcdfe4460fb158ca5d8c92fc99` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:35:04.463516 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf74f6-507b-4d0d-b74d-d44ceeaa5e0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed482cfcdfe4460fb158ca5d8c92fc99)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:35:04.465436 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf74f6-507b-4d0d-b74d-d44ceeaa5e0a admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:35:04.465724 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1447/2905] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:35:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:35:04.514349 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-f7bf74f6-507b-4d0d-b74d-d44ceeaa5e0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed482cfcdfe4460fb158ca5d8c92fc99'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:35:04.520869 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-f7bf74f6-507b-4d0d-b74d-d44ceeaa5e0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed482cfcdfe4460fb158ca5d8c92fc99'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:35:04.533545 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-f7bf74f6-507b-4d0d-b74d-d44ceeaa5e0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed482cfcdfe4460fb158ca5d8c92fc99'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:35:04.540228 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-f7bf74f6-507b-4d0d-b74d-d44ceeaa5e0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed482cfcdfe4460fb158ca5d8c92fc99'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:35:04.540856 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-f7bf74f6-507b-4d0d-b74d-d44ceeaa5e0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed482cfcdfe4460fb158ca5d8c92fc99'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:35:04.547160 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1459/2906] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:35:04 2022] DELETE /identity/v3/users/ed482cfcdfe4460fb158ca5d8c92fc99 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 16:35:04.719744 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659a5bc-7d55-4d1b-beac-b8344a863c3f service placement] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:35:04.732296 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659a5bc-7d55-4d1b-beac-b8344a863c3f service placement] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:35:04.789307 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aefaa49-e581-42fe-be51-68fee5d1a85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:35:04.790161 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefaa49-e581-42fe-be51-68fee5d1a85a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:35:04.790444 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefaa49-e581-42fe-be51-68fee5d1a85a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:35:04.790720 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefaa49-e581-42fe-be51-68fee5d1a85a admin admin] PATH_INFO: `/v3/projects/501da3536d6c41c5b8838bd643c33b22` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:35:04.791911 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aefaa49-e581-42fe-be51-68fee5d1a85a admin admin] RBAC: Authorizing `identity:delete_project(project_id=501da3536d6c41c5b8838bd643c33b22)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:35:04.793829 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aefaa49-e581-42fe-be51-68fee5d1a85a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:35:04.815466 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:35:04.837723 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7aefaa49-e581-42fe-be51-68fee5d1a85a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '501da3536d6c41c5b8838bd643c33b22', 'request_id': 'req-7aefaa49-e581-42fe-be51-68fee5d1a85a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:35:04.843661 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7aefaa49-e581-42fe-be51-68fee5d1a85a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '501da3536d6c41c5b8838bd643c33b22', 'request_id': 'req-7aefaa49-e581-42fe-be51-68fee5d1a85a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:35:04.850859 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7aefaa49-e581-42fe-be51-68fee5d1a85a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '501da3536d6c41c5b8838bd643c33b22', 'request_id': 'req-7aefaa49-e581-42fe-be51-68fee5d1a85a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:35:04.860892 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7aefaa49-e581-42fe-be51-68fee5d1a85a admin admin] The token cache is being invalidate because project 501da3536d6c41c5b8838bd643c33b22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:35:04.861180 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7aefaa49-e581-42fe-be51-68fee5d1a85a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:35:04.861793 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7aefaa49-e581-42fe-be51-68fee5d1a85a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:35:04.863308 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1448/2907] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:35:04 2022] DELETE /identity/v3/projects/501da3536d6c41c5b8838bd643c33b22 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:35:04.867245 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf74f6-507b-4d0d-b74d-d44ceeaa5e0a admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:35:04.868707 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-f7bf74f6-507b-4d0d-b74d-d44ceeaa5e0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:35:04.882357 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf74f6-507b-4d0d-b74d-d44ceeaa5e0a admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:35:04.940705 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea49a0e-c8ab-4ec4-9a21-80e30a1a89b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:35:04.941831 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea49a0e-c8ab-4ec4-9a21-80e30a1a89b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:35:04.942112 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea49a0e-c8ab-4ec4-9a21-80e30a1a89b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:35:04.942385 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea49a0e-c8ab-4ec4-9a21-80e30a1a89b6 admin admin] PATH_INFO: `/v3/users/25bdbb0019cd4fed8ce5fd37e585a43e` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:35:04.943814 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea49a0e-c8ab-4ec4-9a21-80e30a1a89b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25bdbb0019cd4fed8ce5fd37e585a43e)` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:35:04.946067 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea49a0e-c8ab-4ec4-9a21-80e30a1a89b6 admin admin] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:35:04.998309 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0ea49a0e-c8ab-4ec4-9a21-80e30a1a89b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25bdbb0019cd4fed8ce5fd37e585a43e'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:35:05.000035 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:35:05.007737 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0ea49a0e-c8ab-4ec4-9a21-80e30a1a89b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25bdbb0019cd4fed8ce5fd37e585a43e'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:35:05.021250 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0ea49a0e-c8ab-4ec4-9a21-80e30a1a89b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25bdbb0019cd4fed8ce5fd37e585a43e'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:35:05.028425 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0ea49a0e-c8ab-4ec4-9a21-80e30a1a89b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25bdbb0019cd4fed8ce5fd37e585a43e'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:35:05.029081 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.notifications [None req-0ea49a0e-c8ab-4ec4-9a21-80e30a1a89b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25bdbb0019cd4fed8ce5fd37e585a43e'} {{(pid=80286) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:35:05.035199 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1460/2908] 10.222.0.41 () {58 vars in 1223 bytes} [Fri Jun 3 16:35:04 2022] DELETE /identity/v3/users/25bdbb0019cd4fed8ce5fd37e585a43e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:35:05.202668 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aefaa49-e581-42fe-be51-68fee5d1a85a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:35:05.204156 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7aefaa49-e581-42fe-be51-68fee5d1a85a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:35:05.219179 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aefaa49-e581-42fe-be51-68fee5d1a85a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:35:05.276714 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e5042-30cb-4823-8048-aaadcd50cc2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5aca81affd5542cf9480c6e76c8c5478', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450b64d0f10f4cb8949a26d9eea4aef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:35:05.278083 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e5042-30cb-4823-8048-aaadcd50cc2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:35:05.278364 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e5042-30cb-4823-8048-aaadcd50cc2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:35:05.278640 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e5042-30cb-4823-8048-aaadcd50cc2f admin admin] PATH_INFO: `/v3/projects/ac6dce4269204d12bb126054ca8f8442` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:35:05.279866 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e5042-30cb-4823-8048-aaadcd50cc2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac6dce4269204d12bb126054ca8f8442)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:35:05.281800 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e5042-30cb-4823-8048-aaadcd50cc2f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:35:05.327632 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-fd8e5042-30cb-4823-8048-aaadcd50cc2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac6dce4269204d12bb126054ca8f8442', 'request_id': 'req-fd8e5042-30cb-4823-8048-aaadcd50cc2f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:35:05.333584 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-fd8e5042-30cb-4823-8048-aaadcd50cc2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac6dce4269204d12bb126054ca8f8442', 'request_id': 'req-fd8e5042-30cb-4823-8048-aaadcd50cc2f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:35:05.341447 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-fd8e5042-30cb-4823-8048-aaadcd50cc2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac6dce4269204d12bb126054ca8f8442', 'request_id': 'req-fd8e5042-30cb-4823-8048-aaadcd50cc2f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:35:05.351648 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-fd8e5042-30cb-4823-8048-aaadcd50cc2f admin admin] The token cache is being invalidate because project ac6dce4269204d12bb126054ca8f8442 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 16:35:05.351937 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-fd8e5042-30cb-4823-8048-aaadcd50cc2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:35:05.352564 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-fd8e5042-30cb-4823-8048-aaadcd50cc2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 16:35:05.354107 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1449/2909] 10.222.0.41 () {58 vars in 1232 bytes} [Fri Jun 3 16:35:05 2022] DELETE /identity/v3/projects/ac6dce4269204d12bb126054ca8f8442 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 16:35:15.031525 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=12 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:35:15.348641 dv-cinder-836996-28-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:DELETE=14,keystone:SELECT=66,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 16:39:02.870685 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea49a0e-c8ab-4ec4-9a21-80e30a1a89b6 admin admin] Authenticating user token {{(pid=80286) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 16:39:02.872116 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-0ea49a0e-c8ab-4ec4-9a21-80e30a1a89b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:39:03.005948 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea49a0e-c8ab-4ec4-9a21-80e30a1a89b6 admin admin] Validating token access rules against request {{(pid=80286) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 16:39:03.062318 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0fd41-fe0d-4deb-8a56-9be1477602d6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be901786f8934d559e0bac5d622f97cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c274254e510a468e9acbecc6298301cc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80286) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 16:39:03.062961 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0fd41-fe0d-4deb-8a56-9be1477602d6 service placement] REQUEST_METHOD: `GET` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 16:39:03.063244 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0fd41-fe0d-4deb-8a56-9be1477602d6 service placement] SCRIPT_NAME: `/identity` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 16:39:03.063550 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0fd41-fe0d-4deb-8a56-9be1477602d6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80286) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 16:39:03.065295 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.fernet_utils [None req-3ae0fd41-fe0d-4deb-8a56-9be1477602d6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80286) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 16:39:03.076606 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae0fd41-fe0d-4deb-8a56-9be1477602d6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 16:39:03.078154 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae0fd41-fe0d-4deb-8a56-9be1477602d6 service placement] RBAC: Authorization granted {{(pid=80286) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 16:39:03.088499 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: [pid: 80286|app: 0|req: 1461/2910] 10.222.0.41 () {60 vars in 1365 bytes} [Fri Jun 3 16:39:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 16:39:13.083576 dv-cinder-836996-28-iscsi devstack@keystone.service[80286]: DEBUG dbcounter [-] [80286] Writing DB stats keystone:SELECT=89 {{(pid=80286) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}